Embedded Files
With AI Builder Microsoft has made Power Platform more desirable than ever. It is one of the most complex solution from Microsoft but at the same time it is very easy to use.
In this blog we will create “Form processing” model, and we will see how to use it in Power Apps. Purpose of this blog is to give a taste of AI builder with Power Apps. To understand AI Builder capability in detail please follow Microsoft documentation.
Let’s start our discussed with few prerequisites before working on a demo.
Let’s start our discussed with few prerequisites before working on a demo.
Terminologies
Model – Consider a model as a system or function which will take information as input and will generate output based on machine learning capability. This Model once designed and trained can be used in Power App or in Microsoft Automate.
Training – When we will create a model, we provide a set of sample information and we guide what information we want to pick, for example for Object Detection type of model we give minimum of 15 images and tell system how to process these images. The more sample information we provide the more robust our model become. This process of making our model more robust is called training.
We need an Office 365 subscription, for this learning purpose, any free subscription will be enough. Register for one-month free subscription here
We need a Power App environment in US or Europe region. If your O365 subscription is not in US or Europe region then create new environment with Type as Trial and Region as United States. As shown in image at right. Once environment is ready you will be able to see new environment in your Admin centre. As shown in below image.
Create a CDS instance, go to Data, create DB.
In this blog we will create a model for form processing, we will need minimum of 5 forms to train our model. You can use salary slip, mark sheets, or any set of invoices. You need minimum of 5 documents to train your model. In this blog I will use simple PDF which I generated out of MS Word document. As shown in image at right.
Once we are done with prerequisites, we are halfway through. Now time for action.
Once we are done with prerequisites, we are halfway through. Now time for action.
Go to AI Builder > Build, then select “Form Processing”
Name your model, also explore Example and Best practices section as highlighted in image. Also notice below name “What you will need” section for this type of model it says 5+ documents with the same layouts are required. Click on Create.
Now add the field names which you want to extract from your form, as per my form I added Name and Age, click Next .
Add document, notice highlighted text - format, size and pages limitation.
Click on Add document and select document from local machine, SharePoint or Azure Blob Storage.
Add 5 documents, then click on Analyze.
It will take some time to analyze all documents, it depends on number of documents and the complexity of the document.
Now zoom your document for better clarity and selection, then select “Draw in document” under your field names as shown in below image.
Then draw over the text which you want to capture against the selected field, click done as shown in image at right.
Once done then form should look as shown below, just note that I had only two fields to capture Name and Age.
Click on right arrow icon “>”. You will notice on new form fields will be auto selected, if required do the correction.
In one of the form age is missing, in this case I will select “Field not in document” as highlighted in below image.
Once you confirm fields in all forms then click on "Next", check model summery, then click "Train".
If you provided lot of complex form and you are processing lot of fields then it will take some time and you can close the window and come back after some time to check.
But for me it took only few seconds I can see following message.
If you are clicking on “Go to Details page”, you will directly land on your models detail page as shown below.
If you are visiting back on AI builder then you need to go to AI Builder > Models and then need to select your Model to go to Details page as shown below.
Once you are on details page click on Publish. Then click on Use Model as shown below.
Now you will get two option to use your model, for this blog we will select “New App”.
In some time, you will get a sample app provisioned with your AI Builder control – Form Processor.
For the brevity of this blog we will simply add two label field and try to get Name and Age once a document gets uploaded.
You can notice, I mapped Name field of FormProcessor1 control with the text property of label.
Similarly, I mapped age field of FormProcessor1 control with the text property of corresponding label control.
Now its time to test our Model in Power App, click on play and upload a sample form.
As soon as you upload sample file, by clicking on Analyze button, your model will detect required field and accordingly our labels will get populated.
You can also save this information to SharePoint or SQL server or pass to any other system.
Now it’s your turn to explore other types of Models, try to use in MS Automate, and further process your data as required.
Thank you.
Post relevance date : October 2020
Page updated
Report abuse