Working with environments on the Internet Computer
Written by Remco Sprenkels (rem.codes), Blockchain Developer
This is the first part of a blog series that will go through setting up different environments, CI/CD, managing cycles, and setting up static and dynamic canisters.
Summary
In this post, we are going to set up a development, staging, and production environment that are going to be deployed to the live internet computer network. For each environment we will create a unique identity, this step is optional and a decision I made to prevent accidental code pushed to the wrong environment and to keep a clear scope of who has access rights to deploy to a certain environment.
Make sure you have DFX installed if you want to follow the steps.
Creating identities
With DFX you can easily create multiple identities with the following command;
dfx identity new <identity-name>
When running this command you get prompted to enter a password to protect the generated PEM file. With this guide we disabled the encryption which can be done by providing the --disable-encryption
flag, so the command would be
dfx identity new <identity-name> — disable-encryption
We are creating 3 environments, so 3 identities, I would suggest using a recognizable tag per environment identity such as, “development-identity”, “staging-identity” and “production-identity”.
Creating canisters
Each environment is hosted on the internet computer network, so the first step is creating canisters on the live net. For this, I use the Network Nervous System (NNS).
If you have multiple canisters per environment, create a canister for each one by logging into the NNS, navigate to the “canisters” tab and click the button at the bottom. Do note that you need ICP to get the needed cycles to spin up the canisters. I find the NNS canister management pretty unmanageable if you have multiple canisters. Because of this, I suggest adding the principal of the environment identity right after the canister is created and adding it to the canister_ids.json
so you don’t lose track of the canisters you created and what environment they belong to. This will be explained in the next step.
Referencing the canisters with the environment
After a canister is created there is an option to add a new controller. Here we need to add the environment identity principal. You can get this by running the following commands;
dfx identity use <identity-name>
dfx identity get-principal
Copy and paste this principal to the pop-up you get when you add new controllers and confirm your action.
After you are set don’t forget to add the canister principal to the canister_ids.json with the right tag like so;
{
“foo-canister”: {
“development”: <development-canister-id>,
“staging”: <staging-canister-id>,
“production”: <production-canister-id>
}
}
Setting up the environment networks
The final thing we need to do to round up this setup is referencing the correct canisters to the correct environment canisters. This requires us to add the networks to the dfx.json
{
“canisters”: {
// canister declarations
},
“networks”: {
“development”: {
“providers”: [
“https://ic0.app”
],
“type”: “persistent”
},
“staging”: {
“providers”: [
“https://ic0.app”
],
“type": “persistent”
},
“production”: {
“providers”: [
“https://ic0.app”
],
“type”: “persistent”
}
}
}
After this is added you can specify the network with the deploy
command like so;
dfx deploy --network development
Which targets the correct canisters for the right environment to deploy to.
Something else…
When you have a project that has a bunch of canisters that also handle inter-canister calls you are going to run into an issue, and that is that the canister principals that are referenced in the code need to change according to the right environment. You don’t want to have the production canister do an inter-canister call to the development canister and vice-versa.
First things first
The code for the solution is written in Rust, At the time of writing, I don’t think it is possible to do this with Motoko, the main reason being that we need to write to the filesystem and Motoko doesn’t provide this feature. If you work with Motoko I would suggest doing a hybrid approach and using Rust for generating the environment-specific files.
The solution
Before deployment, we will generate an environment-specific file that holds all the canister principals, with this file we can reference the correct canister to do the inter-canister calls to. This also allows us to set environment-specific settings, but more about that in another post.
Make sure you have Rust installed by following these steps.
To go over all steps I will start with a new project by running the following command in the CLI;
dfx new ic_project
Inside the root of your project create a canister_ids.json
file, I’m using the following content for demo purposes;
{
“foo_canister”: {
“development”: “foo_development_canister_id”,
“staging”: “foo_staging_canister_id”,
“production”: “foo_production_canister_id”
},
“bar_canister”: {
“development”: “bar_development_canister_id”,
“staging”: “bar_staging_canister_id”,
“production”: “bar_production_canister_id”
}
}
Next up navigate to the src
folder by running
cd src
Now we will create a Rust project inside our src
folder by running
cargo new env_file_generator
Add a cargo.toml
to the root of your main project (so not in the env_file_generator
folder) and add the following content;
[workspace]
members = [“src/env_file_generator”]
Now your project structure should look something like this. Only the relevant files are in the overview
|-- root
| |-- src
|-- ic_project
|-- ic_project_assets
|-- env_file_generator
|-- src
|-- lib.rs
|-- Cargo.toml
| |-- canister_ids.json
| |-- Cargo.toml
| |-- dfx.json
Next, we need to install some dependencies that are required for the file generation, to do so we need to change the existing Cargo.toml
inside the Rust project directory.
[package]
name = “env_file_generator”
version = “0.1.0”
edition = “2018”[lib]
path = “src/lib.rs”
crate-type = [“lib”][dependencies]
serde_json = “1.0.79”
str-macro = “1.0.0”
The str-macro
is optional, but I suggest installing it to follow the rest of the guide.
Next up we are going to create a generate.rs
file at the same level as the lib.rs
. Here we need to specify some functions to get the code generation to work.
First I declare a struct where we declare the reference for the canister ids
pub struct CanisterIds {
pub foo_canister_id: String,
pub bar_canister_id: String,
}
here we also to_content
functions that allow us to generate the file content, in the example it’s for Rust-based projects, but this would be the place to make the changes to make it compatible with Motoko.
impl CanisterIds {
pub fn to_content(&self) -> String {
let environment = format!(
"pub const ENVIRONMENT: &str = \"{}\";\r",
get_env()
); let foo_canister_id = format!(
"pub const FOO_CANISTER_ID: &str = \"{}\";\r",
self.foo_canister_id
); let bar_canister_id = format!(
"pub const BAR_CANISTER_ID: &str = \"{}\";\r",
self.bar_canister_id
); format!("{}{}{}",
environment,
foo_canister_id,
bar_canister_id
)
}
}
The next function that we need is one to get the current environment where we want to generate the file for;
pub fn get_env() -> String {
let env = env::var("ENV");
match env {
Ok(_env) => _env,
Err(_) => str!("local"),
}
}
Now we need a function to get the the data from the JSON file to use in the file that we generate;
fn get_json_value(json: &Value, name: String) -> String {
let mut return_value: String = String::from("");
let json_value = json.get(name); match json_value {
Some(value) => match value.get(get_env()) {
Some(v) => match v {
Value::String(_v) => return_value = _v.clone(),
Value::Null => {}
Value::Bool(_) => {}
Value::Number(_) => {}
Value::Array(_) => {}
Value::Object(_) => {}
},
None => return_value = str!(""),
},
None => return_value = str!(""),
}
return_value
}
The last step is to actually create the file based on the canister_ids.json
, which can be done with the following function;
pub fn generate_canister_ids() {
let mut env_path = str!(".dfx/local/canister_ids.json");
if get_env() != str!("local") || get_env() != "" {
env_path = str!("canister_ids.json");
}
let dir = PathBuf::from(env::var("CARGO_MANIFEST_DIR").unwrap());
let file_path = dir.parent().unwrap().parent().unwrap().join(env_path);
let file = File::open(file_path).expect("file should open read only");
let json: Value = from_reader(file).expect("file should be proper JSON"); let canister_ids = CanisterIds {
foo_canister_id: get_json_value(&json, str!("foo_canister")),
bar_canister_id: get_json_value(&json, str!("bar_canister")),
};let _ = fs::write("environment_settings.rs", canister_ids.to_content());
}
So now we actually need to trigger the file generation, we can handle this with cargo test
. We can set the environment in the environment variable so the command will look like this.
So to get the total generate.rs
file
use serde_json::{from_reader, Value};
use std::{
env,
fs::{self, File},
path::PathBuf,
};
use str_macro::*;pub struct CanisterIds {
pub foo_canister_id: String,
pub bar_canister_id: String,
}impl CanisterIds {
pub fn to_content(&self) -> String {
let environment = format!("pub const ENVIRONMENT: &str = \"{}\";\r", get_env());let foo_canister_id = format!(
"pub const FOO_CANISTER_ID: &str = \"{}\";\r",
self.foo_canister_id
);let bar_canister_id = format!(
"pub const BAR_CANISTER_ID: &str = \"{}\";\r",
self.bar_canister_id
);format!("{}{}{}", environment, foo_canister_id, bar_canister_id)
}
}pub fn get_env() -> String {
let env = env::var("ENV");
match env {
Ok(_env) => _env,
Err(_) => str!("local"),
}
}pub fn generate_canister_ids() {
let mut env_path = str!(".dfx/local/canister_ids.json");
if get_env() != str!("local") || get_env() != "" {
env_path = str!("canister_ids.json");
}
let dir = PathBuf::from(env::var("CARGO_MANIFEST_DIR").unwrap());
let file_path = dir.parent().unwrap().parent().unwrap().join(env_path);
let file = File::open(file_path).expect("file should open read only");
let json: Value = from_reader(file).expect("file should be proper JSON");let canister_ids = CanisterIds {
foo_canister_id: get_json_value(&json, str!("foo_canister")),
bar_canister_id: get_json_value(&json, str!("bar_canister")),
};let _ = fs::write("environment_settings.rs", canister_ids.to_content());
}fn get_json_value(json: &Value, name: String) -> String {
let mut return_value: String = String::from("");let json_value = json.get(name);match json_value {
Some(value) => match value.get(get_env()) {
Some(v) => match v {
Value::String(_v) => return_value = _v.clone(),
Value::Null => {}
Value::Bool(_) => {}
Value::Number(_) => {}
Value::Array(_) => {}
Value::Object(_) => {}
},
None => return_value = str!(""),
},
None => return_value = str!(""),
}return_value
}
After the following command is ran a file called environment_settings.rs
should appear in the root of the env_file_generator
folder.
ENV=development cargo test
The steps described here can be found in this repo.
If you find any issue or inconsistency, feel free to DM Remco on discord at rem.codes#1526 or Twitter at https://twitter.com/rem_codes