ROSETTA-THOUGHT IS CONSIDERED ALPHA SOFTWARE. USE AT YOUR OWN RISK.
This project is available open source under the terms of the [Apache 2.0 License](https://opensource.org/licenses/Apache-2.0).
The rosetta-thought repository provides an implementation of the Rosetta API for Thought (THT) based on the reference rosetta-bitcoin implementation for Bitcoin in Golang.
Rosetta is an open-source specification and set of tools that makes integrating with blockchains simpler, faster, and more reliable. The Rosetta API is specified in the OpenAPI 3.0 format.
Requests and responses can be crafted with auto-generated code using Swagger Codegen or OpenAPI Generator, are human-readable (easy to debug and understand), and can be used in servers and browsers.
- Rosetta API implementation (both Data API and Construction API)
- UTXO cache for all accounts (accessible using the Rosetta
/account/balanceAPI) - Stateless, offline, curve-based transaction construction from any P2PKH Address
- Automatically prune thoughtd while indexing blocks
- Reduce sync time with concurrent block indexing
- Use Zstandard compression to reduce the size of data stored on disk without needing to write a manual byte-level encoding
The rosetta-thought implementation has been tested on an AWS c5.2xlarge instance. This instance type has 8 vCPU and 16 GB of RAM.
- Adjust your network settings to the recommended connections.
- Install and run Docker as directed in the Deployment section below.
- Run the
Testnet:Onlinecommand.
To increase the load that rosetta-thought can handle, we recommend tunning your OS settings to allow for more connections. On a linux-based OS, you can run these commands (source):
sysctl -w net.ipv4.tcp_tw_reuse=1
sysctl -w net.core.rmem_max=16777216
sysctl -w net.core.wmem_max=16777216
sysctl -w net.ipv4.tcp_max_syn_backlog=10000
sysctl -w net.core.somaxconn=10000
sysctl -p (when done)
We have not tested rosetta-thought with net.ipv4.tcp_tw_recycle and do not recommend enabling it.
You should also modify your open file settings to 100000. This can be done on a linux-based OS with the command: ulimit -n 100000.
rosetta-thought uses memory-mapped files to persist data in the indexer. As a result, you must run rosetta-thought on a 64-bit architecture (the virtual address space easily exceeds 100s of GBs).
If you receive a kernel OOM, you may need to increase the allocated size of swap space on your OS. There is a great tutorial for how to do this on Linux here.
While working on improvements to this repository, we recommend that you use these commands to check your code:
make depsto install dependenciesmake testto run testsmake lintto lint the source codemake salusto check for security concernsmake build-localto build a Docker image from the local contextmake coverage-localto generate a coverage report
As specified in the Rosetta API Principles, all Rosetta implementations must be deployable via Docker and support running via either an online or offline mode.
YOU MUST INSTALL DOCKER FOR THESE INSTRUCTIONS TO WORK.
Running these commands will create a Docker image called rosetta-thought:latest.
After cloning this repository, run:
make build-local
Running these commands will start a Docker container in detached mode with a data directory at <working directory>/thought-data and the Rosetta API accessible at port 8080.
MODE
Type: String
Options: ONLINE, OFFLINE
Default: None
MODE determines if Rosetta can make outbound connections.
NETWORK
Type: String
Options: MAINNET or TESTNET
Default: TESTNET
NETWORK is the Thought network to launch or communicate with.
PORT
Type: Integer
Options: 8080, any compatible port number
Default: None
PORT is the port to use for Rosetta.
You can run these commands from the command line. If you cloned the repository, you can use the make commands shown after the examples.
When testing either Mainnet or Testnet, please run both the online and offline containers at the same time. Do not run Testnet and Mainnet concurrently.
Uncloned repo:
docker run -d --rm --ulimit "nofile=100000:100000" -v "$(pwd)/thought-data:/data" -e "MODE=ONLINE" -e "NETWORK=MAINNET" -e "PORT=8080" -p 8080:8080 -p 10618:10618 rosetta-thought:latest
Cloned repo:
make run-mainnet-online
Uncloned repo:
docker run -d --rm -e "MODE=OFFLINE" -e "NETWORK=MAINNET" -e "PORT=8081" -p 8081:8081 rosetta-thought:latest
Cloned repo:
make run-mainnet-offline
Uncloned repo:
docker run -d --rm --ulimit "nofile=100000:100000" -v "$(pwd)/thought-data:/data" -e "MODE=ONLINE" -e "NETWORK=TESTNET" -e "PORT=8080" -p 8080:8080 -p 11618:11618 rosetta-thought:latest
Cloned repo:
make run-testnet-online
Uncloned repo:
docker run -d --rm -e "MODE=OFFLINE" -e "NETWORK=TESTNET" -e "PORT=8081" -p 8081:8081 rosetta-thought:latest
Cloned repo:
make run-testnet-offline
rosetta-thought uses the syncer, storage, parser, and server package from rosetta-sdk-go instead of a new Thought-specific implementation of packages of similar functionality. Below you can find an overview of how everything fits together:
To speed up indexing, rosetta-thought uses concurrent block processing with a "wait free" design (using the channels function instead of the sleep function to signal which threads are unblocked). This allows rosetta-thought to fetch multiple inputs from disk while it waits for inputs that appeared in recently processed blocks to save to disk.
Before validation, it is important to note that rosetta-thought can use prefunded accounts to automate testing (It currently doesn't, a prompt will appear to fund an address at check:construction). If choosing to utilize prefunded accounts, new accounts will have to be set for testing by modifying rosetta-cli-conf/testnet/config.json for either or both Mainnet or Testnet. Information on how to obtain necessary information can be found within the prefunded accounts section.
- In order to have funds returned to the sending account (minus fees), you MUST set the environment variable for the receiving address within the terminal running the rosetta-cli:
export RECIPIENT=\"receiving_address\"
To validate rosetta-thought, install rosetta-cli and run one of these commands:
Testnet commands
rosetta-cli check:spec --configuration-file rosetta-cli-conf/testnet/config.json- This command validates that the API implementation is working under Coinbase specifications. This will only complete if both the online and offline implementation is running.rosetta-cli check:data --configuration-file rosetta-cli-conf/testnet/config.json- This command validates that the Data API information in thetestnetnetwork is correct. It also ensures that the implementation does not miss any balance-changing operations.rosetta-cli check:construction --configuration-file rosetta-cli-conf/testnet/config.json- This command validates the blockchain’s construction, signing, and broadcasting. When testing, fund the account asking for 1000 tTHT and wait till test completion. Do not fund any of the other accounts manually, the test will do this for you
Mainnet commands
rosetta-cli check:spec --configuration-file rosetta-cli-conf/mainnet/config.json- This command validates that the API implementation is working under Coinbase specifications. This will only complete if both the online and offline implementation is running.rosetta-cli check:data --configuration-file rosetta-cli-conf/mainnet/config.json- This command validates that the Data API information in themainnetnetwork is correct. It also ensures that the implementation does not miss any balance-changing operations.rosetta-cli check:construction --configuration-file rosetta-cli-conf/mainnet/config.json- This command validates the blockchain’s construction, signing, and broadcasting. When testing, fund the account asking for 1000 THT and wait till test completion. Do not fund any of the other accounts manually, the test will do this for you
Read the How to Test your Rosetta Implementation documentation for additional details.
WARNING It is never a good idea to save private keys in plain text. Utilizing prefunded accounts will do exactly this. Using prefunded accounts is not neceessary for testing. Performing the following steps can potentially lead to loss of account and their associated balances.
To retrieve private keys:
- Open ThoughtCore and open the debug console from the tools menu.
- Type the command:
listunspent 1to find an address with atleast 1000 THT for the test. - Type the command:
dumpprivekey "address"to retrieve the WIF encoded private key. - Base58Decode the private key into bytes.
- Remove the first byte (network byte) and the last 5 bytes (compression byte + checksum).
- The remaining string of bytes is the raw private key for the specified address.
You can find the Rosetta API documentation at rosetta-api.org.
Check out the Getting Started section to start diving into Rosetta.
Our documentation is divided into the following sections:
- rosetta-sdk-go — The
rosetta-sdk-goSDK provides a collection of packages used for interaction with the Rosetta API specification. - rosetta-specifications — Much of the SDK code is generated from this repository.
- rosetta-cli — Use the
rosetta-clitool to test your Rosetta API implementation. The tool also provides the ability to look up block contents and account balances.
You can find community implementations for a variety of blockchains in the rosetta-ecosystem repository, and in the ecosystem category of our community site.
This project is available open source under the terms of the Apache 2.0 License.
© 2022 Coinbase © 2022 Thought Network, LLC