Problems getting blocks from Ganache

[nenadjaja]

I tried the following and all of these fail because it can't get the block from the blockchain:

• Adchain-subgraph with --ethereum-rpc and ganache-cli
• Adchain-subgraph with --ethereum-ipc and geth --testnet
• Bitsausage-subgraph with --ethereum-rpc and ganache-cli
• Bitsausage-subgraph with --ethereum-ipc and geth --testnet
  I am getting one of these 2 error msgs pretty much immediately after I run cargo:
  https://github.com/graphprotocol/graph-node/blob/master/datasource/ethereum/src/block_ingestor.rs#L162-L166

Versions:
Ganache CLI v6.1.8 (ganache-core: 2.2.1)
Geth Version: 1.8.14-stable

[Jannis]

We've received another report for Ganache:

Sep 17 15:33:40.443 WARN failed to poll for latest block: ErrorMessage { msg: "could not get block from Ethereum: Decoder error: Error(\"invalid length 1, expected a 0x-prefixed hex string with length of 40\", line: 0, column: 0)" }

being logged a lot. I'm fairly certain this particular error is specific to Ganache. When polling for the latest block in the Block Ingestor, Ganache must be returning something that we are not prepared for.

[Jannis]

Identified one type of error:

Key not found in database

Error message

Sep 17 22:17:18.482 WARN failed to poll for latest block: ErrorMessage {
  msg: "could not get block from Ethereum: RPC error: Error {
    code: ServerError(-32000),
    message: \"Key not found in database\",
    data: Some(Object({\"stack\": String(\"NotFoundError: Key not found in database\\n    at /usr/local/lib/node_modules/ganache-cli/build/cli.node.js:2:773948\\n    at /usr/local/lib/node_modules/ganache-cli/build/cli.node.js:2:776500\\n    at /usr/local/lib/node_modules/ganache-cli/build/cli.node.js:2:748581\\n    at ReadFileContext.callback (/usr/local/lib/node_modules/ganache-cli/build/cli.node.js:2:750214)\\n    at FSReqWrap.readFileAfterOpen [as oncomplete] (fs.js:352:13)\"), \"name\": String(\"NotFoundError\")})) 
  }" 
}

Cause

The same network name (e.g. --ethereum-rpc ganache:...) was used with two different chains (e.g. two separate instances of Ganache; or one Ethereum mainnet and one testnet endpoint).

Why is the error thrown?

Doing this results in the Graph Node having an invalid latest block for this network. It will then get an error from Ethereum when calling eth_getBlockByHash with the hash of this block, which belongs to the previous chain but not the one in use at the time of the error.

[timmclean]

Ahhh knew that would come up. We talked about checking for that in #168, but might be worth a new issue just for the UX improvement

Update: filed #377

[Jannis]

Identified another type of error:

Invalid length 1, expected a 0x-prefixed hex string with length 40

Error message

 Sep 17 23:32:10.097 WARN failed to poll for latest block: ErrorMessage {
  msg: "could not get block from Ethereum: Decoder error: Error(\"invalid length 1, expected a 0x-prefixed hex string with length of 40\", line: 0, column: 0)"
}

Cause

Contract creation transactions have no to address. However, Ganache returns "to": "0x0" instead of "to": null. This causes rust-web3's deserialization of the Transaction object to fail (it has to: Option<H160>).

How to reproduce?

Set up Ganache with a migration that creates a contract or two. Make an eth_getBlockByHash request that includes transactions for a block that contains a contract creation transaction:

➜  subgraph git:(master) ✗ curl --insecure http://localhost:8545 \
                               -v \
                               -H "Content-Type: application/json" \
                               -d '{"jsonrpc":"2.0","method":"eth_getTransactionByHash","params":["0xc9e41a85969b440b161ece21e81bc7ce01d99a5fb01c9f754d56e51614a0de54"],"id":106}'
* Rebuilt URL to: http://localhost:8545/
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8545 (#0)
> POST / HTTP/1.1
> Host: localhost:8545
> User-Agent: curl/7.54.0
> Accept: */*
> Content-Type: application/json
> Content-Length: 142
>
* upload completely sent off: 142 out of 142 bytes
< HTTP/1.1 200 OK
< Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Methods: *
< Content-Type: application/json
< Date: Mon, 17 Sep 2018 22:42:09 GMT
< Connection: keep-alive
< Transfer-Encoding: chunked
<
{"id":106,"jsonrpc":"2.0","result":{"hash":"0xc9e41a85969b440b161ece21e81bc7ce01d99a5fb01c9f754d56e51614a0de54","nonce":"0x03","blockHash":"0xc151b5fc66c4f108f466fb3068bf0d3f09e72014d45dca8302a57cbcf558b08d","blockNumber":"0x04","transactionIndex":"0x00","from":"0xe207259495d9de39e92e8b280a85b4d845906ce6","to":"0x0","value":"0x0","gas":"0x6691b7" ...

The last line above has the incorrect "to": "0x0".

The Ethereum JSON-RPC API spec states that to should be null in case of a contract creation.

[leoyvens]

@Jannis perhaps the best way to fix that last one is to patch web3 to be more flexible? Turns out Ganache is the problem here.

[Jannis]

@leodasvacas Let's find out what's going on here first.

[Jannis]

There are two possible routes here:

1. A fix in Ganache to return "to": null.
2. Custom deserialization for the to field in rust-web3, translating "0x0" to None.

[leoyvens]

Given that Ganache is the side not conforming to the spec, I suppose the right thing to do would be to patch them.

[timmclean]

I filed issue ConsenSys-archive/ganache#907 in the ganache project repo about getting "to": "0x0" instead of "to": null