Category: Azure

Posted on

Connected Field Service with Azure IoT Hub – The Setup

In today’s post, we are going to set up “Connected Field Service” with Azure IoT Hub. A quick recap from the last two posts: 

  1. We started this series by having a look at the raison d’etre of using IoT with Dynamics 365 Field Service
  2. Next, we took a satellite view of various approaches with respective technology stacks 
  3. In this third post, we are going to get a little more hands-on as we will start working on the first approach with Azure IoT Hub.

Pre-Requisites

  1. Dynamics 365 CE tenant with Field Service 
  2. Azure subscription

Above could be trial instances and may not necessarily need to be on the same tenant. In other words, this setup will work even if you have Azure and Dynamics 365 on two different tenants. 

Side note: this second point is really cool because it takes away the dependency of setting up the two services on the same tenant. For instance, you could use your standard Dynamics 365 instance with a free-tier Azure subscription to explore this feature. 

The Setup: 

In Dynamics 365, open ‘Connected Field Service’ app and click on Settings. Click on the ‘Link to Deployment App’. It may ask for permissions for an app to access Azure and Dataverse, click  ‘Accept’. 

This ‘Deployment App’ is an excellent wizard-based tool as it hides underlying technical complexities and simply asks few questions to provision and set up Azure resources and make them ready to talk to Dynamics 365 environment. In the first screen, we need to select Dynamics 365 environment, agree to T&Cs and click Next. 

Next, we need to connect to Azure subscription. If it is on the same tenant, it will go and bring up your subscriptions otherwise you can sign in with a different login. Select the correct Azure subscriptions and click Next

In this step, leave everything as it is and click Next. Basically, we are going with default options and since we are setting up these resources in Azure for the first time, we don’t need to ‘use existing resources’. Click Next. 

Magic is happening

…may take couple of minutes or more

Oh one last thing, as all the azure resources are deployed – next we need to authorise Azure to talk to Dynamics 365. Click on the button: 

It will open up ‘API Connection’ in your Azure portal. Click on ‘Edit API connection’

Click on Authorise 

That will basically enable Azure to talk to Dynamics 365 using that authorised connection 

..and that’s it. But before we finish today’s adventure let’s quickly see what did ‘Deployment App’ do behind the scenes. While you are in Azure, click on All Resources (from the Home screen). This will show you all the resources that are added.

As you can see, it’s a combination of various Azure products but first and foremost, we have Azure IoT Hub. This is going to be the engine of IoT related capabilities. We then have some Logic Apps to run both way comms between Azure and Dynamics 365. API connections are there to support this data traffic and then we have other resources for storage, analytics etc.

I hope this sound very exciting. In the next post, we will have a look at IoT Hub and see how does comms work between IoT devices and Dynamics 365 (via Azure). 

Thanks for reading!

Posted on

Dynamics 365 Field Service and IoT – The Complete Guide

Whenever I talk to customers or within #Dynamics365 community, one of the most FAQs is about IoT (Internet of Things). Nothing garners as much interest as this one. Is IoT just a buzzword or a reality? How does it work with Field Service – or why in the first place IoT has anything to do with Field Service? Does it require any specialized skill set?

Photo by Ruiyang Zhang on Pexels.com

I intend to cover all about IoT and Dynamics 365 Field Service in this series of post. This has been in my mind for some time but I was waiting for the release plan for 2021 Release Wave 2 to ensure we cover some upcoming features as well. Now before we start, just to make scope clear (pun intended) my goal is to:

  • Address the philosophy before jumping on to nuts & bots: why IoT, different approaches, tech stack etc.
  • Present a walk-through of technology nuts & bolts of various IoT approaches
  • Share a future roadmap for further learning and specialisation

… and the only pre-requisite here is familiarity with Dynamics 365 as a platform.

The Series

Here is a series of posts that are part of this guide:

  1. Introduction
  2. Technology Stack and Approaches
  3. CFS with Azure IoT Hub – The Setup
  4. CFS wit Azure IoT Hub – Device Registration and Monitoring [TBD]
  5. Azure IoT Central – The Setup [TBD]
  6. CFS with Azure IoT Central [TBD]
  7. New Features [TBD]
  8. FAQs and Further to explore [TBD]

Why IoT with D365 Field Service?

Let’s start with answering the fundamental question:

What is the basic premise of using Field Service in IoT?

Every business having equipment and assets (think utilities, facility management, LGAs etc.) wants zero down-time i.e. those equipment, devices and assets should be up and running without ideally any outage. This ambition is realized by having two kinds of maintenance:

  • Preventative Maintenance: Every equipment comes with a recommended service calendar. We all know about cars: they need to be serviced every few thousand miles/kilometres or few months. This is called preventative maintenance because it preempts issues or breakdowns by having key components checked, serviced or replaced regularly. It statistically doesn’t bring outages down to zero but it does reduce the possibility significantly.
  • Reactive Maintenance: Once equipment stops working, then fixing it asap, troubleshooting, finding a workaround or making a permanent fix etc. – all comes under the heading of reactive maintenance. In other words, this is ‘break-fix’

Beside others, a key difference between these two types of maintenance is in the way their business process starts:

  • Preventative Maintenance starts with having a service calendar of assets and then following it properly,
  • Reactive Maintenance begins with a call, ticket or an email from customer (somebody will notice that equipment is not working and then call for break-fix)

‘Somebody’ and ‘notice’ in Reactive Maintenance are keywords because they are weak links in the entire process. If somebody makes a mistake and doesn’t notice a faulty equipment, or identify the wrong equipment, or does notice but report late etc. – then equipment’s overall ‘up-time’ will reduce. This prompts a question: can we optimize this process or in other words, instead of reactive, can we have a proactive maintenance? The answer is: yes!

  • Proactive Maintenance: When an equipment/device/asset sends an alert signal identifying (a) an issue or (b) a deviation from standard values, then troubleshoot and fix the potential problem.

Since signals would be caught automatically (without manual noticing), investigation and break-fix would happen earlier. Many times such signal identify a potential problem which may happen few days, weeks or months down the line. This makes Proactive Maintenance a key part of services and maintenance portfolio in every industry and therefore IoT (signals, alerts from equipment) works with Dynamics 365 Field Service.

Hope this makes a case for ‘Why’ behind this series. In this next post, we’ll talk about the ‘How’. That is, we will look at various IoT deployment models and approaches with Dynamics 365 Field Service.

Thanks for reading!

Feel free to share your feedback, I’d love to hear your thoughts 😊

Posted on

Making Field Service Smarter using AI Builder

Quality audit is part of a work management life cycle in most Field Service organizations. To ensure customers are satisfied, equipment is up and running, and there are no SLA breaches, a quality audit is conducted in all or statistically sampled work orders. This often means creating another booking, or a task, or creating a follow-up work order for quality check and of course additional resource requirements – but is there a smarter solution possible in Dynamics 365 Field Service? This post explores the possibility of using AI Builder.

The Challenge:

In our Contoso field service organization, this is how the work happens:

  1. Field Technicians are booked to conduct the work
  2. Field Technicians complete the work, take a picture and uploads in the app
  3. Supervisors take a look at the picture, make a conscious call to either approve or visit a site to confirm work is properly completed
  4. Work Order is completed after the quality check has passed

For the point (3) above, we’ll use AI Builder’s object detection model to train it on pictures of quality vs faulty work, and then once a model is trained, hook it up with Field Service to check the quality of work and then change the status of Work Order automatically. This is how the process will look like:

The Setup:

We’ll need the following Power Platform products:

  1. Dynamics 365 Field Service
  2. AI Builder
  3. Power Automate Flow

All of the above can be setup as a trial for this post.

As for the images of quality vs faulty work, I’ve tapped into vast datasets available on Kaggle and found this one with the images of Lego bricks. For our experiment today, we’ll use 20 images each of 4×4 vs 1×2 bricks to differentiate quality vs fault work:

Faulty Work:

Quality Work:

The Play:

Following are three steps to run this play:

  1. Dynamics 365 Setup
  2. Build a model in AI Builder
  3. Create a flow to call AI Builder model with Dynamics 365

1. Dynamics 365 Setup:

Create a new field on Work Order entity:

Name: Completed Work

Type: File

….and drop it on a form.

Field Technicians upload the image of completed work in this field. We’ll use Power Automate to call AI Builder model on this field’s content.

2. Build a model in AI Builder

Create a new AI Builder model of type Object Detection and name it as ‘Quality Check’. Click Next..

Select ‘Common Object’ and click Next..

Add two ‘Object names’ as Quality and Faulty work. Click Next.

Click on Add images and upload all 40 images..

Click on Add images and upload all 40 images..

Click on Tag images and start tagging images one by one as Quality…

and Faulty work:

Let’s have a look at the Summary and then click on Train. It will take few minutes.

Once trained, run a few Quick Test. Seems to be working for me 🙂.

Quick Test 1:

Quick Test 2:

Create a flow to call AI Builder model with Dynamics 365

Let’s go to Power Automate flow and click New. We will use ‘Common Data Service (current environment)’ trigger and have the following actions:

Trigger: When a record is created, updated or deleted on Work Order

Steps:

  1. Get file from Work Order’s file field
  2. Predict action to call our AI model and pass on the image
  3. Based on the prediction, either mark Work Order as Closed or send an email to Field Technician

Here is our Power Automate flow:

..the second step:

the third and following steps…

All done! Start the play by uploading the image of completed work (in this case 1×2 or 4×4 lego brick images). The flow will run, send the image to AI model, and update the status of Work Order or send the email. Successful output:

Viola! That was fun and what we just experimented with is, in fact, a game changer in terms of making enterprise applications (like Dynamics 365 Field Service) really smarter – coupled with the fact that you can get this up and running in a very short timeframe and without expensive solution development/deployment.

Hope this was useful. If you have any suggestions/questions, please do reach out – would love to hear from you.

Posted on

Developing heart risk calculator using AI Builder

The idea of this article is to experiment with AI Builder to develop an app for self-assessment of heart-related diseases. An AI-powered based app could tap into historical data and identify patterns, and then use them to assess the user’s entered data to predict if there is a risk of heart diseases or not.

The app is not to be used for medical advice or prescription, rather this is an experiment to use Prediction model of AI Builder with a #nocode approach.

The first step to accomplish this was to find data on heart-related risks. Luckily I got it, thanks to Kaggle. You can go here (it is one of the many publicly available datasets) and download the data.

  1. Next, goto Power Apps maker portal, create a solution and create an entity for storing heart diseases statistics. Create a model-driven app and add the entity into the app.
  1. Import data into the entity
  1. With data imported into the system, click on AI Builder on the left. Click on Prediction
  1. A new window will open, give a name to your model
  1. Select entity and the field which holds prediction outcome
  1. Select fields that contribute to the outcome (for example, de-select created on, modified on etc. if they are selected)
  1. Next, you can filter the data to be analyzed. In this case, we just selected ‘Skip this step’ and clicked ‘Next’
  1. View the summary and click on Train
  1. This data will be then fed into training the model
  1. Once training completes, publish the model. Accuracy is pretty impressive.
  1. Once published, details screen will show (a) option to run model by clicking on ‘Use Model’, (b) percentage distribution of individual field’s contribution to the outcome, and (c) how model is being used (currently it automatically syncs the outcome with the database daily).

It is important to note here that unlike other AI models, Prediction models run daily. If you want to run it on demand, click on Use Model and then click on Run Now.

  1. Now lets’s go back to the app and create a new user profile.
  1. Now either wait for Prediction model to run or force it to run now (step 9 above).
  2. Once the model has run, go back to your record. Here is a snapshot of my data:

The model has predicted user does not have the risk of heart diseases, viola!

Now that I have seen this running, I can already look back and think of past implementations where this model through AI Builder can add a lot of value. What do you think? Please feel free to share your feedback on Twitter.

Happy AI Building!

Posted on

Building Crowd Counting Model using AI Builder

Counting the number of people in a photo or video feed is one of the interesting problems in the world of AI, and for a good reason because “number of people/given area” is a useful metric for number of practice purposes. For example:

  1. Retail: Knowing foot traffic in a specific store area or floor indicates how impactful the discount promotion is, or how popular a particular brand/product is
  2. LGAs/councils/city governments: Lot of smart cities want to know near real-time information on how many people are in a particular public space, park, or tourist spot to ensure adequate resources are available to ensure city services are functioning correctly
  3. Police/law enforcement/emergency services: Knowing the crowd size, helps emergency services to deploy resources appropriately

Also worth mentioning, like any other use of AI models, this is an area where functioning under the theme of AI for good is very important.

Data and tool

To experiment how AI Builder can help in crowd counting with #nocode approach, the first to-do is to secure data. Kaggle is one of the best resources for publicly available data. You can search and find a lot of data sets related to any topic. I found this one interesting and downloaded the data set on my laptop.

The tool is an AI Builder Object Detection model.

Let’s dive in…

  1. Goto Power Apps maker portal and click on AI Builder on the left. Click on Object Detection (it is under preview at the moment).
  1. A new window will open, give a name to your model
  1. You can select objects from CDS or your local machine.
  1. Select device..
  1. ..and add images (downloaded previously from Kaggle dataset). Images will upload
  1. Mention which object(s) you want AI model to auto-detect. In our case, this is only ‘Person’
  1. Next, we need to identify the object (Person – in our case) in each of the images. Click on each image..
  1. and tag all the persons
  1. AI Builder screen will show your progress on this tagging exercise
  1. When all the images are tagged, click on ‘Done tagging’
  1. Click Next
  1. This data will be then fed into training the model
  1. When the training will complete (may take a few minutes), it will show the degree of accuracy

It is not ideal but is not too bad for a #nocode experiment. Click on Publish. There are two ways to use this model; in a Power Automate Flow or in a Power Apps app. For this experiment, we will create a quick Canvas App. Click on Power Apps.

  1. This will open a familiar Power Apps maker portal. Add a button with the label ‘Analyze’ and a Data table.
  1. Behind a button, set output of the AI model to a ‘Persons’ collection
  1. Bind Data table with ‘Persons’ collection
  1. Save and experiment. Select image, let the AI model run and click on Analyze.

The model is returning the number of objects (persons) it identified with quite an impressive accuracy.

Please do let me know your feedback, always love to hear back from #PowerApps community. Happy Power-apping!

Posted on

Identify key words in Power Platform 2020 Release 1 plan using AI Builder

This is a part-fun and part-experimenting AI Builder article. Microsoft released 2020 Release Wave 1 Plan for Power Platform back in January. It is full of exciting stuff; all the new features, enhancements and updates planned circa April this year — and it is long read; 149 pages to be exact. Now, what if we could run some sort of ‘smart algo’ on this document to identify all the keywords (oft-repeated words) in it! This will give a new angle into what is the recurring theme, oft-repeated objectives and important buzz words that are mentioned in the release plan. Now before we go into details of what was that ‘smart also’ and how it was achieved, here is the results of the exercise:

Very insightful, right? And as you may have guessed, the ‘smart algo’ here is AI Builder. Let’s get our hands dirty…

  1. Goto Power Apps maker portal and click on AI Builder on the left.
  1. Click on Key Phrase Extraction model
  1. Click on ‘Use in a flow’
  2. This will lead you to Power Automate flow creation screen. Add a manual trigger and add ‘Text’ input
  1. A key limitation currently with Key Phrase Extraction model is that it supports the analysis of 5120 characters only, so we divided total length of the text with 5120 and then ran a loop to call Key Phrase Extraction model repeatedly, pass a substring of fewer than 5120 characters from the complete text and then save all the keywords in one variable. Add Predict action, select Key Phrase Extraction model and pass on a substring of complete text (less than 5120 characters). Add an action to send keywords using email
  1. Run the flow….
  1. ..and copy-paste Release Plan text and click on Run Flow
  1. Flow successfully runs, repeatedly calls Key Phrase Extraction AI model and sends all the identified keywords.

Note: The approach does return duplicates because in each iteration it processes 5000 characters, I had to de-duplicate result manually in excel but can’t see any reason why same can’t be done using Power Automate flow.

This was a quick demonstration of the power of Key Phrase Extraction AI model. In terms of its real-world usage, it could be used to review customer feedback, case descriptions, partners communications etc. to identify the theme or key elements of the conversations, which could be then addressed appropriately. Hope you find it useful, if you have suggestions or questions, please feel free to share.

Posted on

Automatically process Uber Receipts for expense approvals using AI Builder

A common scenario in most of the businesses is for employees to collect their receipts of the expenses made and then submit expenses along with receipts at the end of the month (or week/fortnight) for reimbursement. This creates a long queue (aka processing time and possible delay in processing) for HR/accounts team to review each receipt for approval. In this article, I have experimented with AI Builder to see if it can help in making the process faster and easier.

An example scenario here considers submission of an expense through a model-driven Power Apps app, but this could be used in Canvas Power App, Dynamics HR, emails or even any other system that connects with CDS. The process is to auto-approves the expense in expense in record tallies with the ‘Total’ field in Uber receipt or sends for further approval (manual offline process).

This demonstration requires three components to be built:

  1. A model-driven Power Apps app with an expense entity and form
  2. An AI Builder model to process Uber receipts
  3. A Power Automate flow to run on the create/update of expense, trigger AI Builder model and based on the result, change expense status

Step 1. Model-driven Power Apps app
Create a new PowerApps model-driven app and create an entity ‘Travel Expense’ entity with three fields:
a. Title
b. Total Expense ($ value)
c. Status

Note: An entity should support notes and attachments.
Also note: this step is not discussed in detail in this app. If model-driven apps are new for you, have a look at this excellent Microsoft docs resource.

Step 2. Create an AI Builder Model
Goto Power Apps maker portal and click on AI Builder on the left.

Click on Form Processing. You’ll notice the Examples and Best practices tab which have great tips on setting up AI models. Enter the name of your model (Read Uber Receipts – for example) and click on Create.

Next screen will ask you to upload documents. Select the files and upload.

Once the documents are uploaded, click on Analyze.

Now the AI engine will run..

..and it will ask you to tag the fields which it was able to find in the documents.

Select the Total field on the document and label it.

Click ‘Done’ and click ‘Next’

Next click on Train to train the newly created model

Once the training is completed..

..go to the Details page. This shows the document that is used to train the AI model, field identified during form processing and how to use the model (Power Automate and Power Apps).

You can also do the quick test by uploading another document and let the AI model find data in the document. Publish the model.

Step 3. Create Power Automate Flow

Go to the solution (created in Step 1) and click on New-> Flow to create a new flow.

Create a flow with a CDS trigger on the entity that we created in Step 1. Next, find notes associated with the expense record, fetch attachment in Notes and then use ‘Apply to each’ to run an AI model on each file. For calling the AI model, use ‘Predict’ action.

Next, fetch the result from the AI model and compare with the expense entered by the user in the form. If both match, then approve the record, else send for manual review.

Lastly, let’s test it out:

  1. Create a new expense record and attached an Uber receipt:

.. in the background, the flow ran:

Record gets updated to manual approval process (In Progress) if expense record total doesn’t match total in receipt:

..otherwise it gets auto-approved:

Hope you will find this useful. Please feel free to share your feedback 🙂