Run model by using CinnBuilder API

In this tutorial, we will introduce the ways to build and run a model using CinnBuilder APIs.

import cinn
from cinn import frontend
from cinn import common
import numpy as np
# sphinx_gallery_thumbnail_path = './paddlepaddle.png'

Define the CinnBuilder.

Using CinnBuilder is a convenient way to build a model in CINN. You can build and run a model by invoking CinnBuilder’s API as following.

name: the ID of CinnBuilder

builder = frontend.CinnBuilder(name="add_conv")

Define the input variable of the model.

The input variable should be created by create_input API. Note that the variable here is just a placeholder, does not need the actual data.

type: the data type of input variable, now support Void, Int, UInt, Float, Bool and String, the parameter is the type’s bit-widths, here the data type is float32.

shape: The shape of the input variable, note that here does not support dynamic shape, so the dimension value should be greater than 0 now.

id_hint: the name of variable, the defaule value is “”

a = builder.create_input(
    type=common.Float(32), shape=(1, 24, 56, 56), id_hint="A")
b = builder.create_input(
    type=common.Float(32), shape=(1, 24, 56, 56), id_hint="B")
c = builder.create_input(
    type=common.Float(32), shape=(144, 24, 1, 1), id_hint="C")

Build the model by using CinnBuilder API

For convenience, here we build a simple model that only consists of add and conv operators. Note that you can find the operator’s detailed introduction in another document, we won’t go into detail here.

Generally, the API in CinnBuilder is fine-grained operator, in other words, the algebra or basic DL operator.

d = builder.add(a, b)
res = builder.conv(d, c)

Set target

The target identified where the model should run, now we support two targets:

DefaultHostTarget: the model will running at cpu.

DefaultNVGPUTarget: the model will running at nv gpu.

if common.is_compiled_with_cuda():
    target = common.DefaultNVGPUTarget()
else:
    target = common.DefaultHostTarget()

print("Model running at ", target.arch)

Out:

Model running at  Arch.X86

Generate the program

After the model building, the Computation will generate a CINN execution program, and you can get it like:

computation = frontend.Computation.build_and_compile(target, builder)

Random fake input data

Before running, you should read or generate some data to feed the model’s input. get_tensor: Get the tensor with specific name in computation. from_numpy: Fill the tensor with numpy data.

tensor_data = [
    np.random.random([1, 24, 56, 56]).astype("float32"),
    np.random.random([1, 24, 56, 56]).astype("float32"),
    np.random.random([144, 24, 1, 1]).astype("float32")
]

computation.get_tensor("A").from_numpy(tensor_data[0], target)
computation.get_tensor("B").from_numpy(tensor_data[1], target)
computation.get_tensor("C").from_numpy(tensor_data[2], target)

Run program and print result

Finally, you can run the model by invoking function execute(). After that, you can get the tensor you want by get_tensor with tensor’s name.

computation.execute()
res_tensor = computation.get_tensor(str(res))
res_data = res_tensor.numpy(target)

# print result
print(res_data)

Out:

[[[[11.571602  10.037912  12.110612  ... 11.803285  10.4503975
    12.39268  ]
   [12.118155  11.331117  11.2125    ... 10.547782  12.086964
    10.297881 ]
   [11.0286255 10.750516  11.5008745 ... 12.0907955 12.7944145
    12.78049  ]
   ...
   [10.9749365 10.080397  10.170232  ... 11.073846  13.112092
    11.807571 ]
   [11.376714  13.390822  10.701132  ...  8.375713  12.820743
    12.455052 ]
   [13.08021   10.476735  12.054475  ... 11.040751  10.412604
    12.462891 ]]

  [[11.715233  10.517147  11.598742  ... 11.481009   9.9403715
    11.722999 ]
   [11.467173  10.645823  11.332581  ...  9.847761  10.552914
    10.138514 ]
   [10.465738   9.978148  10.484111  ... 10.257616  12.233998
    12.2045965]
   ...
   [10.829692   9.717082   9.959618  ... 10.864071  13.141303
    11.720429 ]
   [11.869147  12.892423  11.600269  ...  9.288053  13.1629715
    11.902034 ]
   [12.196988  10.507401  12.522007  ... 11.306245  11.8908
    11.879142 ]]

  [[ 7.0207853  6.780528   7.1823373 ...  7.7550783  7.077457
     8.6607685]
   [ 7.534683   7.4253626  7.140933  ...  6.2216945  8.172223
     6.5323215]
   [ 7.563627   7.8048253  7.6544023 ...  7.292347   8.270231
     8.02856  ]
   ...
   [ 7.4650517  6.3309436  6.0289545 ...  7.835066   8.898898
     7.6010804]
   [ 8.542815   8.381237   6.937894  ...  6.03162    8.595205
     8.731508 ]
   [ 8.340959   6.5326405  7.241155  ...  7.484935   7.0709844
     7.9060907]]

  ...

  [[11.478321  10.483002  10.756042  ... 11.76803   10.395924
    12.380386 ]
   [12.15006   11.723205  11.261727  ... 11.094739  12.16321
    11.7729025]
   [10.40236   11.365674  12.381306  ... 12.539725  13.496915
    13.967277 ]
   ...
   [12.309769  10.177728  10.188444  ... 12.48512   13.499703
    12.724175 ]
   [13.344845  13.365545  12.548767  ...  9.05944   15.144086
    13.429598 ]
   [15.571053  11.193485  13.457433  ... 13.745621  12.467106
    13.37518  ]]

  [[13.315583  13.315888  12.039522  ... 11.5344515 12.055772
    13.386152 ]
   [12.634843  11.162848  12.577147  ... 11.380696  12.809536
    12.337452 ]
   [12.297415  11.702893  12.784704  ... 12.3519335 14.45426
    13.621389 ]
   ...
   [12.130316  11.363695  11.692377  ... 12.95829   14.201069
    12.42412  ]
   [13.981301  13.729186  12.080306  ...  9.493916  14.731629
    13.593263 ]
   [14.004305  10.771607  12.4516535 ... 14.381553  12.467554
    13.803441 ]]

  [[13.910039  14.278668  13.928883  ... 14.040823  12.109072
    14.80458  ]
   [13.744701  13.308964  14.76345   ... 13.136091  14.855599
    14.121593 ]
   [14.864944  12.936687  15.485008  ... 14.28767   16.60473
    15.257153 ]
   ...
   [13.914206  11.639483  12.371835  ... 14.61134   16.152748
    14.045977 ]
   [16.510437  16.021467  13.342856  ... 10.186195  16.333027
    15.138903 ]
   [15.972713  12.521988  15.687452  ... 15.2209425 14.478452
    13.770096 ]]]]