Documentation

Documentation in a phased development

Version

  • Created: 2022
  • Update: 2022

Blockchain IC

The Internet Computer (ICP) is the first World Computer blockchain, powered by innovations in cryptography and computer science derived from deep crypto research begun in 2015. It is a blockchain that is scalable without limits and can host and run smart contracts with efficiency comparable to traditional centralized IT infrastructure. Internet Computer smart contracts can even serve web experiences, securely interact with Web2, and create and sign transactions for other blockchains. As a World Computer blockchain, it supports the full end-to-end decentralization of online systems and services, which can now be built entirely on chain exclusively from smart contracts, without the need for centralized traditional IT and Big Tech's cloud services.

Database

Difibase - NoSQL database management system on the Internet Computer You can create your own database cluster and have several databases in it. It is possible to create a simple database and with file support. It is also allowed to integrate their own database versions and use them in this service

Architecture

The main feature of the Internet Computer blockchain is canisters. Canisters perform the following functions: data storage and (or) computational tasks. This project is based on these principles. There are several entities in which the principles of functioning are laid down. This is about the same as in classic databases. Here they are: administrator, cluster, database.

User interaction with objects via the web interface. Objects in this context are: clusters and databases

A distinctive feature is modularity. There is a possibility to choose between different versions of databases. You can also add your own database source code. Compile it in .wasm file send it to wasm-storage. (details below)

At the moment, there are two versions of the database in the project.

Their sources:

with ordinary:
https://github.com/fury02/Difibase/tree/main/difibase-0.0.44/src/dbs/databases/version_easy
with support files:
https://github.com/fury02/Difibase/tree/main/difibase-0.0.44/src/dbs/databases/version_files

They are compiled into db_easy.wasm and db_files.wasm

Scheme install wasm

Canister creation scheme and wasm installation

Install wasm file

Download the ready wasm-file.

For example, it will be cluster.wasm (this is a linking file for several instances) and db_easy.wasm

Upload(install) cluster.wasm

Upload(install) db_easy.wasm

We will need these files in the future when creating a cluster and instances. Note: if the system already has these files, there is no need to download wasm files. Unless you want to use your own specific (custom) implementations

Login with Plug

For further work, a Plug wallet is required. Web address: https://plugwallet.ooo/. After downloading the wallet, perform authentication using the Plug. Other methods are not supported yet.

Cluster create

Further actions require replenishment of the wallet with ICP coins. The main thing is never to store funds on exchanges and, if possible, use hardware wallets

Let's create our own cluster (by default)

Value ICP = 0.3 Name = "Cluster"

Most likely, the amount of ICP is quite small.

The fact is that we need about 1200000000000 cycles (or 1.2T cycles) Take a look at the picture below. The actual cost of conversion is also indicated there.

The stages of creation are shown below in the screenshots

1)

2) Be sure to wait until the operation is completed! Do not refresh the page until the operation is completed!

3)

4)

You can also create a specific cluster with a different version specified in wasm-storage

You can stop the cluster or resume work. Details below on the screenshots.

1)

2)

Database create

At the moment, wasm-storage contains two wasm objects: db_easy.wasm; db_files.wasm . By default, instance is created from db_easy.wasm. Create from db_files.wasm . Below are the screenshots.

Select cluster

Wasm file db_files: version "1"; name "db_files.wasm"

1) Create instance:

2) Check balance

3) It is also necessary to select the amount ICP . Let me remind you about 1200000000000 cycles (or 1.2T cycles)

4) Be sure to wait until the operation is completed! Do not refresh the page until the operation is completed!

5)

6) New database (instance)

1) Let's go to the next tab. We can update the instances state. (Similar to clusters)

2) Select cluster and instance

1) You can: clear the database, stop, start, or delete. Note: Deleting an instance leaves it registered in the system but removes it from the blockchain. The following shows the stop and start of the database.

2) Starting:

Financing of canisters

Canisters almost always consume cycles. In order for the canister to be always active, there must be a certain number of cycles on the balance of the canister. At the moment, we can convert ICP into cycles. Perhaps in the future it will take place automatically or a notification by mail. On today need to look at the balance of of clusters and database and, if necessary, carry out financing.

Amount cycles in selected cluster

1) Let's choose cluster financing

Be sure to wait until the operation is completed! Do not refresh the page until the operation is completed!

2) Minted cycles

Next, we will similarly finance the instance database. Let's choose a database from the available ones. It can be seen that the balance is now: 1.15827T cycles

We will finance the database in the amount of 0.09 ICP

The minted cycles will approximately be in the amount of 248292000000

1)

Be sure to wait until the operation is completed! Do not refresh the page until the operation is completed!

2)

3)

Easy database

Parametrs:
table_key - name (or key) table
column_key_name - name (or key) column
column_key - name (or key) column
row_key - the key of the table row is actually entity_key
entity_key - key the main key of the recorded data. It can be a number and or a string. In fact, this is the ID of a row in the table.
entity_value - recorded value

- Creating a table, adding data, adding a column, or updating data:

replace_value (

table_key: Text,

column_key_name: Text,

entity_key: Text,

entity_value: Text

) : async ?Text;

The added value or null is returned

Note:
- if there is no table, then it is created
- if there is no column, it is created

- Delete entity cell value:

delete_table_cell_value (

table_key: Text,

column_key_name: Text,

entity_key: Text

) : async Bool;

Returns the result of the operation (true or false)

- Delete table rows value:

delete_table_entity (

table_key: Text,

row_key: Text

) : async Bool;

Returns the result of the operation (true or false)

- Delete a column in a table:

delete_column (

table_key: Text,

colunm_key

) : async Bool;

Returns the result of the operation (true or false)

- Delete table:

delete_table (

table_key: Text

) : async Bool;

Returns the result of the operation (true or false)

- Find table entity cell:

find_table_cell (

table_key: Text,

column_key_name: Text,

entity_key: Text

) : async Text;

Returns a value or an empty string

- Find row:

find_table_cell (

table_key: Text,

entity_key: Text

) : async Text;

Returns a value or an empty string

- Get table values (json):

get_table_entityes (

table_key: Text

) : async Text;

Returns a value(json) or an empty string ([{}] or "")

- Get table values (collection):

get_collection_table_entityes(

table_key: Text

) : async [Text];

Returns a array values

- Get tables in database(json):

get_tables() : async Text;

Returns a value(json) or an empty string ([{}] or "")

- Get tables (collection):

get_collection_tables() : async [Text];

Returns a array values

- Get name tables in database(json):

get_table_keys(

table_key: Text

) : async Text;

Returns a value(json) or an empty string ([{}] or "")

- Get name tables in database (collection):

get_collection_table_keys(

table_key: Text

) : async [Text];

Returns a array values

- Clear table:

clear_table(

table_key: Text

) : async Bool;

Returns the result of the operation (true or false)

- Clear column in table:

clear_column(

table_key: Text,

colunm_key: Text

) : async Bool;

Returns the result of the operation (true or false)

**Нou can view a more complete and up-to-date API: Difibase

Web

Coins

After creating a cluster and database instances in it, we can test them. Let's go to the next tab.

Select instance

Fill in some data (BTC)

Fill in some data (ETH)

Fill in some data (ICP)

Next, let's look at the table

Similarly filled in the "Price" column

Let's go to the next tab

Upload the file

1)

2)

We can see in the table below how the link to the file was created. This is the Guid 0e99c15d6e7642e08d80cd6a01fb37a2

Go ahead and download the file

1)

2)

3)

Consider the last operation in this example. These are not all operations that can be tested in the Web interface. The Web interface also contains only a part of the API.

Pre-filled information

Remove the "true" value in the Stable column of Tether and USDC

1)

2)

The "Usage" tab contains: #1 Databases and #2 Databases (files)

#1 Databases - this is a simple database with no file operations, so it is not supported in #2 Databases If a database with file support is installed, it is supported in both "#1 Databases " and "#2 Databases"

Console

Planets (earlier)

In this material there is a canister of the mxjrx-tiaaaa-aaaah-aaaah-cai service,
it can change if you have your own solution, then this one will be a different ID

Let's consider a simple example, using the example of the planets of the solar system.
As an experiment, we want to create such a simple table:

We can execute such a command using an agent in a web application or a command line. The command line will be used here.

dfx canister --network=ic call mxjrx-tiaaa-aaaah-aaoxq-cai replace_value '("Planets", "Planet","3","Earth")'
Where: "Planets" - table name; "Planet" - colunm name; "3" - id; "Earth" - planet name;

Having previously initialized the agent, the pseudocode string will have the following form
ic_agent.replace_value("Planets", "Planet","3","Earth")
Add the radius column with ID equal to 3. Thereby linking the data.
replace_value '("Planets", "Radius (km)","3","6371")'
Let's change some data and add another planet - Mars.
replace_value '("Planets", "Planet","4","Mars")'
replace_value '("Planets", "Radius (km)","4","3389")'
Visually, in our view, the data table should look like:

By executing the command
dfx canister --network=ic call mxjrx-tiaaa-aaaah-aaoxq-cai get_table_entityes '("Planets")'
We get the following result. In fact, it is an array of json objects. But we can also get a pure json object.

The following api command returns a json object that can then be deserialized.
dfx canister --network=ic call mxjrx-tiaaa-aaaah-aaoxq-cai get_table_entityes_json '("Planets")'
Result:

Generally speaking, the ID field implies meaningful information. But it's okay if you use random values.
This implementation has the concept that you know or semantically (mathematically) understand the values of the identifier.
(Several implementations are planned for the user's choice in future versions) This is typical for NoSQL type databases.
To make it more interesting, I'll add a few more planets. And we will get all the keys (more precisely, the ID) to this table.
replace_value '("Planets", "Planet","2","Venus")'
replace_value '("Planets", "Radius (km)","2","6051")'
replace_value '("Planets", "Planet","7","Uranus")'
replace_value '("Planets", "Radius (km)","7","25362")'

We will get the keys by commands. Below, the json object can be obtained as an array. Both possibilities are presented.
dfx canister --network=ic call mxjrx-tiaaa-aaaah-aaoxq-cai get_table_keys_json '("Planets")'
dfx canister --network=ic call mxjrx-tiaaa-aaaah-aaoxq-cai get_table_keys '("Planets")'

*(the key as well as the ID have an equivalent value)
In this case, identifiers have a semantic meaning. These are planets in order from the sun.
But it was possible to number by auto-increment or enter any value, including the name of the planet itself.

Let's say we don't know anything about the current data structure. Next, let's search for values.
dfx canister --network=ic call mxjrx-tiaaa-aaaah-aaoxq-cai get_tables
dfx canister --network=ic call mxjrx-tiaaa-aaaah-aaoxq-cai get_tables_json
The result of the query of all tables:

Realizing that this is a planet table, we can find out what the radius of the seventh planet is. Below is the request itself.
dfx canister --network=ic call mxjrx-tiaaa-aaaah-aaoxq-cai find_table_cell '("Planets", "Planet","7")'

A search was performed on all cells of the specified table
Below is a request to get the whole string.
dfx canister --network=ic call mxjrx-tiaaa-aaaah-aaoxq-cai find_table_value '("Planets","7")'

It is not yet possible to create external links between tables.
You can easily compensate by expanding the table dynamically. Yes, this creates some redundancy, but so far only so.
Let's say we need to create a sign of the presence of a chemical element - let's say water.
Water is definitely there - true, definitely not - false and an indefinite null situation. Below is a modified data table.

The actual addition of a column (Water) by calling a function with the necessary parameters:
dfx canister --network=ic call mxjrx-tiaaa-aaaah-aaoxq-cai replace_value '("Planets", "Water","3","true")'

From the query, we see that updated data has returned, those rows that have not been updated have the values "null".
We will correct this situation and move on.
replace_value '("Planets", "Water","4","true")'
replace_value '("Planets", "Water","2","true")'
replace_value '("Planets", "Water","7","true")'

Let's add another column, similar to the example considered. Visually it will look like this:

Now you can experiment with the last two columns.
To begin with, clear the "Water" column (You can also clear the entire table, details in the API)
dfx canister --network=ic call mxjrx-tiaaa-aaaah-aaoxq-cai clear_column '("Planets", "Water")'

Undefined or null values are uninformative, so we decide to completely remove the column from the Planet table
dfx canister --network=ic call mxjrx-tiaaa-aaaah-aaoxq-cai delete_column '("Planets", "Water")'
From the query, we see that the "Water" column has disappeared

Deleting columns is not a frequent activity, let's completely delete the row with the planet Uranus.
Below is the command and the result of the operation performed.
dfx canister --network=ic call mxjrx-tiaaa-aaaah-aaoxq-cai delete_table_entity '("Planets", "7")'

Similarly, you can delete data in a separate cell. Now let's try to clear the entire table and look at the result.
dfx canister --network=ic call mxjrx-tiaaa-aaaah-aaoxq-cai clear_table '("Planets")'
The request get_table_entityes_json '("Planets")' will return an empty array or a json object equal to "[ ]"
The get_tables request shows us that the table has remained in the database.

In order to completely delete the table, we use the following query.
dfx canister --network=ic call mxjrx-tiaaa-aaaah-aaoxq-cai delete_table '("Planets")'

Sevice info.

Next, we will clear the entire database of information. We see that the occupied memory has not decreased at least, but in fact it should remain as well.
This behavior is due to the fact that the canister marked the occupied space, but the data was actually erased.
With the following entries, the size will not increase to the actual marked-up space.
Only after overcoming it will it begin to grow and is displayed as increasing.

Integration

- To create an "agent" and an "actor" to interact with your database, it is recommended to familiarize yourself with link and link.

- A large number of examples: link.

- Interaction with the actor is also present in this project. (See the example above) service-difibase/src/util/actors/ic_network (Source code from the links at the very beginning.)

It is recommended to use React-Native to develop and add "actors" to the mobile application. In this case, there will be no differences from the web application. It is also possible to add to an existing application written in native Android (Kotlin; Rust) But these are more special cases and there is no universal solution. You can use the developer forum to clarify link, or write to me at furysafik@gmail.com

Security

- At the moment there is no protection against loss of integrity of information in the database (changes (modification) of information by third parties).

- There is no ready-made (developed) version of information encryption. You can encrypt the data and store the information in the database in encrypted form. A separate end-to-end encryption solution may be developed in future.

Changelog

Version 0.0.2 (December, 2022)

  • AddedUpdated basic information

Version 0.0.1(2022)

Initial Release

End.