Merge pull request #363 from subsquid/misc-fixes-by-abernatskiy-feb16

Misc fixes by abernatskiy feb16
subsquid · Feb 15, 2024 · 4246b02 · 4246b02
2 parents 461e9c2 + 27ca27f
commit 4246b02
Show file tree

Hide file tree

Showing 9 changed files with 232 additions and 184 deletions.
diff --git a/docs/cloud/overview.md b/docs/cloud/overview.md
@@ -16,8 +16,8 @@ The deployment is managed by the file `squid.yaml` in the root folder of the squ
 Make sure to check our [best practices guide](/cloud/resources/best-practices) before deploying to production!
 :::
 
-:::warning
-Yarn is not supported. Use `npm` to install Squid CLI and manage your squid's dependencies.
+:::tip
+Consider our [RPC proxy service](/cloud/resources/rpc-proxy) if your squid requires an endpoint.
 :::
 
 ## 0. Install Squid CLI

diff --git a/docs/cloud/reference/rpc-proxy.md → docs/cloud/reference/rpc-proxy-networks.md b/docs/cloud/reference/rpc-proxy.md → docs/cloud/reference/rpc-proxy-networks.md
@@ -1,18 +1,22 @@
 ---
 sidebar_position: 35
 title: addons.rpc section
-description: Built-in caching RPC endpoints
+description: Request a built-in RPC service
 ---
 
-# RPC proxy
+# addons.rpc manifest section
 
-Since the Arrowsquid release, Subsquid Cloud provides a built-in caching RPC proxy service. Currently, it is a premium feature, with plans to make a subset of calls available to free tier users.
-
-## Available networks
+Enable the add-on like this:
+```yaml
+deploy:
+  addons:
+    rpc:
+      - eth-goerli:http
+```
 
 <details>
 
-<summary>EVM</summary>
+<summary>Available EVM networks</summary>
 
 | Network name            | network:protocol             |
 |-------------------------|------------------------------|
@@ -53,7 +57,7 @@ Since the Arrowsquid release, Subsquid Cloud provides a built-in caching RPC pro
 
 <details>
 
-<summary>Substrate</summary>
+<summary>Available Substrate networks</summary>
 
 | Network name | network:protocol           |
 |--------------|----------------------------|
@@ -86,27 +90,4 @@ Since the Arrowsquid release, Subsquid Cloud provides a built-in caching RPC pro
 
 </details>
 
-Enable the RPC add-on like this:
-```yaml
-deploy:
-  addons:
-    rpc:
-      - eth-goerli:http
-```
-
-## Processor configuration
-
-With the add-on successfully enabled, your squid will get a unique proxied endpoint to the requested network. Subsquid Cloud will make its URL available to the deployed squid at the `RPC_${Upper(network)}_${Upper(protocol)}` environment variable. Assert it to avoid compilation errors. We also recommend rate limiting RPC proxy requests on the processor side to the same rate as was used in the manifest: 
-```ts
-import {assertNotNull} from '@subsquid/util-internal'
-
-processor.setRpcEndpoint({
-  // dash in "eth-goerli" becomes an underscore
-  url: assertNotNull(process.env.RPC_ETH_GOERLI_HTTP),
-  rateLimit: 10
-})
-```
-
-## Pricing
-
-RPC proxy requests are priced [at a flat rate](/cloud/pricing/#rpc-requests), with substantial packages included for free for all [organization types](/cloud/resources/organizations). Pricing does not depend on the call method.
+See [RPC proxy](/cloud/resources/rpc-proxy) guide for more details on the service.
diff --git a/docs/cloud/resources/rpc-proxy.md b/docs/cloud/resources/rpc-proxy.md
@@ -0,0 +1,75 @@
+---
+sidebar_position: 70
+title: RPC proxy service
+description: Built-in RPC endpoints
+---
+
+# RPC proxy service
+
+Subsquid Cloud provides a built-in RPC proxy service. The service is available to all squids deployed to the Cloud, although [playground](/cloud/resources/organizations/#playgrounds) squids only have access to a limited number of calls.
+
+[Professional organizations](/cloud/resources/organizations/#professional-organizations) do not have that limitation and [can access](#external-access) their RPC proxy endpoints from outside of the Cloud.
+
+Enable the RPC add-on in the [manifest](/cloud/reference/manifest) like this:
+```yaml
+deploy:
+  addons:
+    rpc:
+      - eth-goerli:http
+```
+See the [reference page](/cloud/reference/rpc-proxy-networks) for a list of available networks and their aliases (`eth-goerli:http` in this example).
+
+## Processor configuration
+
+With the add-on successfully enabled, your squid will get a unique proxied endpoint to the requested network. Subsquid Cloud will make its URL available to the deployed squid at the `RPC_${Upper(network)}_${Upper(protocol)}` environment variable. Assert it to avoid compilation errors. We also recommend rate limiting RPC proxy requests on the processor side to the same rate as was used in the manifest: 
+```ts
+import {assertNotNull} from '@subsquid/util-internal'
+
+processor.setRpcEndpoint({
+  // dash in "eth-goerli" becomes an underscore
+  url: assertNotNull(process.env.RPC_ETH_GOERLI_HTTP),
+  rateLimit: 10
+})
+```
+
+## External access
+
+[Professional organizations](/cloud/resources/organizations/#professional-organizations) can [access their RPC proxies from outside](https://app.subsquid.io/rpc) of the Subsquid Cloud.
+
+Among other things, this enables a development process that uses the proxy for both local and Cloud runs transparently. Here an example configuration for Ethereum Mainnet:
+
+<details>
+
+<summary>Here is an example configuration for Ethereum Mainnet</summary>
+
+1. Add the following variable to `.env`:
+   ```bash
+   # get the url from https://app.subsquid.io/rpc/chains/eth
+   RPC_ETH_HTTP=<your_external_proxy_url>
+   ```
+
+2. Enable the RPC proxy in squid.yaml:
+   ```yaml
+   deploy:
+     addons:
+       rpc:
+         - eth:http
+   ```
+   This will add and populate the `RPC_ETH_HTTP` variable for Cloud deployments.
+
+
+3. Configure the processor to use the URL from from `RPC_ETH_HTTP`:
+   ```ts
+   import {EvmBatchProcessor} from '@subsquid/evm-processor'
+   import {assertNotNull} from '@subsquid/util-internal'
+
+   export const processor = new EvmBatchProcessor()
+     .setRpcEndpoint(assertNotNull(process.env.RPC_ETH_HTTP))
+     // ...the rest of the processor configuration
+   ```
+
+</details>
+
+## Pricing
+
+RPC proxy requests are priced [at a flat rate](/cloud/pricing/#rpc-requests), with substantial packages included for free for all [organization types](/cloud/resources/organizations). Pricing does not depend on the call method.
diff --git a/docs/sdk/resources/basics/external-api.md b/docs/sdk/resources/basics/external-api.md
@@ -40,7 +40,7 @@ async function getETHPriceByDate(timestamp: number): Promise<bigint> {
 
 ## IPFS fetching
 
-For reliable indexing of content stored on IPFS (e.g. NFT metadata) we recommend fetching from dedicated IPFS gateways, e.g. provided by [Cloudflare](https://www.cloudflare.com/web3/).
+For reliable indexing of content stored on IPFS (e.g. NFT metadata) we recommend fetching from dedicated IPFS gateways, e.g. provided by [Filebase](https://docs.filebase.com/ipfs/about-ipfs/ipfs-gateways).
 
 For a more elaborate example of with IPFS gateway and external API calls, inspect the [Step 3 of the BAYC tutorial](/sdk/tutorials/bayc/step-three-adding-external-data).
 

diff --git a/docs/sdk/subsquid-vs-thegraph.md b/docs/sdk/subsquid-vs-thegraph.md
@@ -16,7 +16,7 @@ The data transformation and presentation is enabled by a growing list of tools a
 - Typescript based [Squid SDK](/sdk/) offers pluggable data targets for both online and offline use-cases
 - Python-based [DipDup SDK](https://dipdup.io/docs/quickstart-evm?ref=blog.subsquid.io)
 - Subgraphs (via [Subsquid Firehose adapter](/subgraphs-support/))
-
+- ApeWorx (via [ape-subsquid plugin](/apeworx))
 
 ## Feature matrix
 
@@ -25,7 +25,7 @@ The data transformation and presentation is enabled by a growing list of tools a
 |  Programming language           |     Typescript            |    AssemblyScript (complied to WASM)     |
 |  Indexing speed                 |     ~1k-50k bps           |       ~100-150 bps                       |
 |  ABI-based generator            |        Yes                |          Yes                             |
-|  Real-time indexing (unfinalized blocks)   |     Yes (*)    |          No                              |  
+|  Real-time indexing (unfinalized blocks)   |     Yes        |          No                              |  
 |  Off-chain data                 |        Yes                |        No                                |
 |  Data targets                   |     Customizable          |      Postgres-only                       |
 |  Customizable DB migrations     |        Yes                |        No                                |
@@ -38,13 +38,8 @@ The data transformation and presentation is enabled by a growing list of tools a
 |  Subscriptions                  |  Yes                      |       Via middleware                     |
 |  Hosted service                 |  Yes                      |       Yes (to be sunset)                 |
 |  Secret env variables           |  Yes                      |          No                              |
-|  Payment                        |  Fiat, subscription-based (**) |   GRT, pay-per-query                |
-|  Decentralization               |  Opt-in, via decentralized data targets (Kwil DB, Ceramic) |  The Graph network   |
-
-
-(*) to be supported by the Squid SDK in Q1 2023
-
-(**) for the Subsquid Cloud Premium plan 
+|  Payment                        |  [Fiat, subscription-based](/cloud/pricing) |   GRT, pay-per-query                     |
+|  Decentralization               |  Decentralized data sourcing via [Subsquid Network](/subsquid-network), with opt-in decentralized data targets (Kwil DB, Ceramic) and processing (via Lava, in development)  |  The Graph network   |
 
 ## Architecture difference
 

diff --git a/docs/sdk/tutorials/bayc/step-three-adding-external-data.md b/docs/sdk/tutorials/bayc/step-three-adding-external-data.md
@@ -193,7 +193,7 @@ URI retrieval here is similar to what we did in the exploration step: we create
 
 ## Retrieving external resources
 
-In the `fetchTokenMetadata()` implementation we first classify the URIs depending on the protocol. For IPFS links we replace `'ipfs://'` with an address of an IPFS gateway, then retrieve the metadata from all links using a regular HTTPS client. Here for the demonstration purposes we use the public [ipfs.io](https://ipfs.io) gateway, which is slow and prone to dropping requests due to rate-limiting. For production squids we recommend using a dedicated gateway, e.g. from [Cloudflare](https://www.cloudflare.com/web3/).
+In the `fetchTokenMetadata()` implementation we first classify the URIs depending on the protocol. For IPFS links we replace `'ipfs://'` with an address of an IPFS gateway, then retrieve the metadata from all links using a regular HTTPS client. Here for the demonstration purposes we use the public [ipfs.io](https://ipfs.io) gateway, which is slow and prone to dropping requests due to rate-limiting. For production squids we recommend using a dedicated gateway, e.g. from [Filebase](https://docs.filebase.com/ipfs/about-ipfs/ipfs-gateways).
 
 ```typescript
 export async function fetchTokenMetadata(