In this post, I’ll be talking a bit about my final university project to get my bachelor’s degree. But first, there’s something that really bothers me, which I’d like to address first.
Terminology mishaps
I recently tweeted that I finally got my Bachelor of Science certificate, although I had a little bit of a trouble with picking names. First, I call it a “Bachelor’s degree in Information and Communications Technology”.
Then, I call it a “Bachelor of Science” for a “Computer Science” program.
At some point I also started calling the institution I attended a “college”, but it seems that it’s actually called a “university of applied sciences”.
This inconsistency is mainly because of the education system being in Dutch, and it’s hard for me to find internationally accepted equivalent terminologies to what I’ve attended. In order to make things more official, I looked at my certificate’s addendum because there, everything’s written in an internationally accepted English format. Handy if I ever decide to move to a foreign country one day.
To make this finally official, once and for all: I got my Bachelor of Science certificate for the study “Information Technology” at a “university of applied sciences”. I have already thanked my instructors, family as well as friends, for their (emotional) support and guidance. I’m not sure if I would’ve made it, if it wasn’t for their encouragement.
In simpler words: I can officially write software professionally.
Anyway, the final project
In order to graduate, I had to work on what the university calls a “Final Project”. This was basically an internship of 5 months, working on an original assignment at a software development company. The final project at the company can be described as an Internet of Things (IoT) project:
“Delve into the inner workings of Azure IoT Hub and create a proof of concept (PoC) application making use of this service, to prove the power of Cloud IoT-technology; fast deployment and little overhead for the developers.”
The PoC that I had to develop, can be described as following: “Work on an IoT solution called ‘De Slimme Sportpaal’ (‘The Smart Sporting Pole’). The device can be deployed in public parks and the likes where people in proximity can ‘sign in’ using a dedicated mobile app, in order to jog through a predetermined route. Based on people’s activity and performance, they’ll get recommendations about other similar people whom they can meet up with. They can then jog together and have a jolly time!”
This PoC combined with the description of the final project is quite a broad use case, as there are multiple technologies involved. What I had to do is basically this:
- My solution had to be cloud-native, so I had to look into Microsoft Azure, as I never worked with its services before.
- I had to look into the Azure IoT Hub specifically, to understand its inner workings.
- I had to look into developing IoT hardware. I had no experience with this, as my study is “Software Development”. IoT leans more towards “Embedded Software Development” which is another study at the university.
- I had to look into mobile app development. Until before the internship, I’ve only worked with native Android, native iOS, as well as Xamarin.Forms apps. All of these only once.
- I had to design the system, kind of “gluing” together all of the above.
In the past, it was required to write a research paper for your assignment. This time around, they discarded that requirement, allowing me to be fully creative with my solutions.
Project Plan
According to my university, the project plan is the first thing you hand in to the company as well as your instructors. Officially speaking, I had 5 weeks to get my project plan in order. This was organized as follows:
- 2 weeks for the initial draft.
- 3 weeks for the final version.
After handing in the initial draft, I had 3 more weeks to finish the final version. This also included waiting for feedback from the instructor and assessor.
My initial draft was the complete project plan, without any missing chapters. I spent 2 full weeks on researching the specifics of the project, as well as researching what a good project plan would look like. Then I applied this knowledge to my draft. I used the university’s template but it’s that: just a template. The content I had to think up myself.
I’ve written the project plan keeping the “client’s point of view” in mind. Clients have questions and concerns that need to be answered clearly and explicitly on paper. I took this to heart, to the point of actually treating the project plan as a contract, between the client and me as a developer. I’ve written the project plan, trying to cover every little detail:
- Company background.
- Stakeholders of the project and their description (e.g. the client, the end-users).
- Services and their description (e.g. Microsoft Azure).
- Project goal(s): What’s the problem which lead to this project? How is the problem going to be solved (i.e. what is the project)?
- Project results: What will be handed in at the end of the project? A list of documents and artifacts as well as their description. The list also contains the receiving parties. Due to the university-context, I also mentioned that the university would receive certain documents and artifacts for grading.
- Project milestones: General list of dates and/or quantized days with milestones (e.g. deadlines, amount of days off, etc.) as well as dismissing possible, optimistic assumptions (e.g. I made it perfectly clear that I’ll assume the “worst case scenario” of the project plan indeed taking 5 weeks).
- Conditions and agreements: Conditions to work on the project properly (e.g. services, a company account, additional costs during the project) as well as agreements regarding the actual act of working on the project (e.g. working hours, calling in sick), as well as my communication with the client (e.g. progress reports).
- Artifacts and quality: Reference to the “Project results” from earlier, but with the focus on artifact quality requirements, actions to produce said artifacts, and process quality. I also added extra artifacts, such as my thesis and my thesis presentation, with the university as the “client” for those.
- Software development process: Short descriptions of possible software development methods (e.g. scrum) and the process of choice, including the reasoning why. The process I choose and possible modifications to this are then further elaborated upon. I also mention crucial tools and services (e.g. Azure DevOps for version control and iteration plans).
- Project organization and communication: List of members of this project (stakeholders, instructors) and their contact information. As I’m the sole developer of the project, I also added the standard meeting days and times with the project members.
- Planning: General timeline of the project based on the chosen software development method, as well as milestones and if applicable, the general activities within a typical iteration.
- Risks: Practically speaking, the project plan should cover everything in such detail that there are no more risks, save for the extreme ones I have no control over (e.g. certain online services being down).
- Sources: It’s nice for the client to understand what a “Twin Peak Model” is. Having sources makes your project plan and claims look more credible.
It’s a huge list, I know. But apparently I worked so meticulously on the plan, that one of the instructors at the university decided to use the draft (!) of my project plan as a positive example, for his students. That was very flattering to hear.
Microsoft Azure and The Cloud™
Azure is a Cloud service provided by Microsoft. It runs on dozens of data centers which are distributed across the globe. Azure offers many “building blocks” (i.e. services) to developers, from Virtual Machines to literally just functions (i.e. “serverless”). Interestingly enough, these two building blocks are on the complete opposite ends of the Cloud spectrum:
The concept of “The Cloud” was new to me. Before I started this final project, I never developed things using the Cloud before. In fact, when I thought of the Cloud, I thought about programs such as Dropbox, OneDrive, or this video. This was just the consumer’s point of view though. Consumers don’t (normally) think about how Dropbox works. In my case, I just imagined it’s a bunch of hard drives somewhere connected to the internet and that’s it. There is some truth in this but the Cloud seems to be more than just storage.
The Cloud is continuously expanding and evolving. Look at the amount of services Azure has. Although I have no service growth numbers for Microsoft Azure, here are some for Amazon Web Services. The amount of major features in AWS grew from 159 to 1017 in a matter of five short years. The sky is the limit with the Cloud at the developer’s disposal (pun not intended).
The Cloud makes it easy to develop solutions. I already briefly talked about Virtual Private Servers in a previous post. In my final project, I used services which did more than just hosting a server and data. The major ones I will discuss below.
Azure IoT Hub
Azure IoT Hub is a service which allows the developer to connect IoT devices to Microsoft Azure. It actually serves like some sort of a gateway which can take on really heavy traffic; we’re talking about millions of simultaneously connected devices with millions of events per second.
Try building a server and network which can take on those numbers. It’s too challenging and expensive and you’ll probably just end up DDOSing your network instead unless you make it really robust. My project didn’t require this kind of performance but it had to make use of Azure IoT Hub anyway, to prove a point; that it can be used in production easily for IoT projects.
IoT devices can connect with Azure IoT Hub by making use of the Azure IoT Hub SDKs. They support various languages, allowing a high range of devices to make use of Azure IoT Hub.
As I mentioned earlier, Azure IoT Hub serves as a gateway to Microsoft Azure. Data sent from IoT devices (for example, weather stations) are ingested by the hub. Then, other Azure services, such as Azure Streaming Analytics or Azure Functions can consume the telemetry for further processing using a pub/sub pattern. Which brings us to…
Azure Functions
Azure Functions is the “serverless” solution of Microsoft Azure. Serverless, very shortly described, are basically functions with certain triggers which run in the Cloud. Triggers can vary from simple timers to HTTP requests. The developer doesn’t need to set up or maintain a server (such as Node.js or ASP.NET); Microsoft Azure does this for you. This means that Microsoft Azure takes care of all the overhead, such as software updates, operating system updates, server updates, resource allocation, and so on. All the developer has to do, is writing functions and deploying them.
Azure Functions are tightly integrated with many services of Microsoft Azure. It can receive data directly from Azure IoT Hub or other Azure services. It can also output data to various services, such as Cosmos DB. All I have to do is add some connection strings and some references in the code. I don’t even need to set up the connection manually most of the time; it happens automatically, by making use of attributes, saving me the trouble of writing boilerplate code.
In my project, I mainly used Azure Functions to ingest telemetry from my prototype IoT device, as well as a tiny REST API serving as an abstraction layer over my database of choice. Which brings us to…
Azure Cosmos DB
Azure Cosmos DB is one of the few Cloud database solutions of Microsoft Azure. Azure Cosmos DB stores data in a format called the “atom-record-sequence“, and has 5 APIs which can access said data in different formats: SQL, Document (MongoDB), Key-Value, Graph and Columnar. This is defined on deployment-level, however. You can’t have one deployment use multiple APIs.
The interesting thing about Azure Cosmos DB is that it is schema-less; data (as well as metadata, such as modification timestamp) are stored in JSON-format, meaning you can have nested documents, and each document could differ in format, even if you select the SQL API.
Finally, Azure Cosmos DB can be somewhat of an enterprise/heavy-duty service. It can duplicate data across data centers across the globe meaning the latency to access the data would be low and the data would be continuously backed up.
Designing and validating the architecture
After looking into Azure IoT Hub and how to process its data, the next step for me was finding the right building blocks to design the architecture as well as the integration with my smartphone app.
The process was iterative due to the project management method I chose: Rational Unified Process. Rational Unified Process can be divided into four phases: inception, elaboration, construction and transition. The elaboration phase is used to validate the architecture and having a skeleton/template to use in the construction phase. Although the phases consist of iterations, my process was actually pretty linear.
I worked “decision-centric”, meaning that one decision was based on another. I kept a timeline of decisions that I took, as well the problems they addressed in what they call a “decision chronology view”:
The blue blocks are the iteration and the phases. The colorful blocks are the decisions: the code, a short description, the decision I took and finally, the date of the decision. The grey blocks are the moments where I validate the decisions I took so far, by writing a prototype application. The prototypes are incremental, meaning that they expand upon previous prototypes I’ve written, just to make sure everything still works.
The decisions themselves were basically mini-researches. Here’s an example, with some of the text trimmed for brevity:
In the decisions, I basically addressed an issue with constraints, listed some alternatives, then started researching those alternatives. Eventually, based on the results and the constraints, I came up with a conclusion and moved on to the next decision. Finally, for good measure, there are references to requirements, concerns and sources.
The most crucial part is the “arguments” part, where I write down facts while linking appropriate sources. Because I was working in an online environment, I could place links so that anyone else could just check them easily.
Because I documented my decisions so thoroughly, it is pretty easy for other developers to re-trace my steps. They can also understand why I took certain decisions in my final architecture.
Final architecture
I won’t be describing the full architecture as it’s much more complicated than it actually is, as I also implemented other major features: automatic “over-the-air” updates with a working deployment pipeline, as well as provisioning new IoT devices. The simplest way to describe the “happy flow” of my architecture for regular users is as follows:
- The IoT Device (Raspberry Pi) receives Bluetooth Low Energy data with the user id of the user from the mobile app, and sends it to the Azure IoT Hub.
- Azure IoT Hub sends said data to Azure Cosmos DB, starting or ending a “session” for said user.
- This change is shown in the mobile app in real-time by using Azure SignalR (basically websockets).
- The mobile app has a very simple account system which also makes use of Cosmos DB. Users can create an account and log in/out.
Closing words
This final internship was extremely educational. Throughout the architecture process, I had to research a lot of IoT and Cloud theories, so I learned a lot about both. The satisfying part is that I came up with the architecture on my own. However, I think the architecture has the potential to be even better. If I had discussed it every now and then with senior colleagues, I think we would find some improvements. The building blocks I used are just a metaphorical drop in the ocean, after all, and I didn’t have the time to study every single drop available to me.
The nice thing about working in the Cloud is that I could fully focus on developing the application, rather than deploying hardware, databases and servers on-premises. Thanks to the Cloud, I managed to create a solution which would probably take over a year to develop from scratch, incurring the necessary costs as well. Not to mention that because I went “serverless”, I don’t have to maintain any servers; Azure automatically updates the used software for me.
After I finished the final project, I took a 4 week break. Ever since last week, I started working as a junior software engineer at the company I did my final project at. To me, it’s definitely a good and exciting start of a new career!