During this sprint, we continued to work on revising and testing our methods that we created. At first, I was responsible for creating the update method as stated in my last blog. I found the PouchDB API to be extremely helpful for this as it essentially gave me what I needed to create the update method. However, we realized that we could save lines of code by having the add method update the data if it already exists. This made more sense so I started working on the getDoc method. Here is a screenshot of the code I wrote:

This code is simple enough, but it was the tests that I struggles with through this sprint:

Even though I failed to get this test to pass *correctly* (it passed but gave errors in the karma console and didn’t work the way I expected), I discovered that I became a lot more comfortable talking about javascript and typescript. Before this class, I did not have any experience learning anything about the angular2 framework and nothing about javascript and typescript. After this weeks sprint, if we had more time I would dive deeper into promises and how to handle them correctly in test suites (it is more complicated than I would think). I would also dive deeper into javascript/typescript since that was the majority of what we used to put this code together.

Our team is still working well together, however I have noticed a slacking in communication between each other. This is only normal as the semester is coming to an end and people are getting ready for finals and presentations and what not. If our team could proceed differently, I think that they would all agree that we were really close to figuring out the solution to our problems. Working with other teams and investigating the code they wrote helped us get another perspective of the code that we wrote. If we had more time we would also clean our code to make it much more readable with clear comments. For example, since we didn’t have the time to learn the correct coding etiquette our add method ended up looking like this (the thens and catches are all promises, which I am sure we could have done differently):

However, even though this code looks really messy and complex, it works and the test passes. One thing that helped our team progress through our struggles was using console.log outputs to see where the code would reach before failing. This was a very useful tactic for us. Another useful tip that helped us get through was the console on the karma testing screen. This had much more information about what was going wrong in our code than the webstorm terminal offered. Webstorm automatically opens the karma browser, you can press “CTRL+SHIFT+I” to bring up this window with the elements, console, source, network and more. You can also view and delete the database that you have created. This proved to be helpful for ensuring that our add method worked even when the tests didn’t.  Here is a screenshot:

This screenshot sums up all of our troubles. In the webstorm terminal the test “passes” but when you open up the console here you can see how many errors it has generated.

After all of this we learned that a better way of going about testing would be to test that our methods are calling pouchDB’s method in the correct order at the correct time. We may have dove a little too deep into specifically testing every little detail about our own methods that it was extremely inefficient. We would definitely change the way we tested our code as it would have saved us some time and headaches.

From the blog CS@Worcester – Rookey Mistake by Shane Rookey and used with permission of the author. All other rights reserved by the author.

From the blog CS@Worcester – BenLag's Blog by benlagblog and used with permission of the author. All other rights reserved by the author.

This week marked the end of the 6th sprint, and with this being the last one to write code and finish up our tasks it was very hectic. It was by far the most challenging sprint and also one of the most productive sprints as well. It was productive because we were able to wrap up the frontend and backend work, the challenges came when putting together the two. Trying to connect those two layers of the application really exposed lots of gaps in my knowledge that I would need to quickly fill up to have any hope of completing the tasks on time. Being more comfortable working on the backend I was able to get those tasks done relatively easily, but the front end is completely foreign. I had to learn about how UI components such as Checkboxes and Buttons work, how they get declared, and most difficult of all how to reference them and their values in the backend code. Working on this has given our team an even better understanding of how things fit together, and had we known that things would be this challenging before, we would’ve begun learning about all of this much sooner.

Like all of the other sprints, the team worked together very well. The work was very evenly divided and we communicated very well. Teamwork was a bigger part of this sprint than most of the others because we were each working on a different layer of the application. Even while working with all of these different moving parts, we were still able to quickly help each other out when errors and other issues came up with the application. Going forward we don’t plan on changing anything regarding teamwork. As for my own personal performance and participation, I think that I did pretty well. I tried to help out my teammates on some issues that they were having and reached out to them on mine.

As for what was done specifically, we added a checkbox that would allow you to choose whether you would save your credentials for later or not. I was in charge of the backend component which was essentially taking the boolean from the checkbox and setting up the logic to have it selectively call our new functionality or not. This had to be done while ensuring that the login logic that was already there still worked perfectly as before. Another one of my team members was in charge the the UI for the checkbox. We also had a team member working on writing some tests for this new functionality. Once that was done, lots of work was done on integrating all of these layers together and making sure that things work like we’d expect. While all of this was going on we also had other teammates working on documenting all of the work that was done, this in order to give a quick and concise summary of exactly what was done and how to use it.

That being said, this was a very successful sprint, there are still a few minor things to iron out but once complete, we will have a great product to show for it!

From the blog CS@Worcester – Site Title by lphilippeau and used with permission of the author. All other rights reserved by the author.

This sprint, my team finally got around to the work of dividing up our service into sections and digging into implementation of code and tests.  While it’s frustrating that it took this long, and that we were unable to import either the crypto-js library or Angular’s native crypto library, it does feel good to have something ready to pass on.

My responsibility was to write the portion of the service that generates a private encryption key and stores it locally.  I communicated with our contact at Ampath in order to work out the details of what should be stored and for how long.  I ended up with the following requirements:

1. The key can only be stored for the duration of a login session.
2. The key needs to be unique to a user.
3. The key needs to be generated in a deterministic way — each time the user logs in, the key must be the same so that they can access records encrypted in a previous session.

I chose to use the second iteration of the password-based key derivation function, implemented in crypto-js (Wikipedia link here).  This function generated a key of a set byte length based on a password, an optional salt, and some number of hashing iterations.  Unfortunately, that’s where I hit a bit of a wall.  For reasons that neither I nor anyone else I worked with (my teammates, members of other teams, and Dr. Wurst) could determine, crypto-js does not want to import properly into an Angular app.  We spent time looking for similar problems in various forums, uninstalling/reinstalling the packages, and making sure the proper typings were installed without any success.

I then tried to work around the problem by converting the crypto-js implementation to typescript  (crypto-js file here, on their github page).  However, this required the use of a custom type that I was unable to get to function as well as the use of bitwise operations prohibited by Ampath’s linter.  I know that Erik, one of my teammates, also looked into alternate libraries but was unable to find anything that he could get to work.

I then moved on to writing unit tests, mocking the effects of an actual encryption algorithm in my code.  While getting these tests passing was fairly trivial with mock encryption library calls, I hope that they will be of use to whoever picks up this part of the project next.

I believe that my team did a good job dividing tasks evenly this sprint, but we really needed to be more active on Slack to communicate problems and progress between work classes, even outside of the standups.  We did communicate well enough to make sure we properly handled our team git repository, and on Thursday we should be able to get it ready to integrate into the class repository in addition to dividing up work on the presentation.

I think the biggest takeaway for me from this sprint is that working with external libraries can be somewhat uncertain.  Even after we researched them and got crypto-js functioning in a spike solution, it didn’t work when we tried to import it into the Ampath app.  If I could do something differently (either this past sprint or earlier), I think I would have chosen to try to implement the encryption algorithms ourselves.  While it might have been harder and more work, that would leave a much easier task of integration with the app.

From the blog CS@Worcester – orscsblog by orscsblog and used with permission of the author. All other rights reserved by the author.

Sprint 6 was a continuation of sprint 5 and hasn’t had much difference. As usual, we started sprint 6 by reviewing what was done in sprint 5, particularly our achievements and failures. Jason updated us with the version of code to work with at that point. Our activities in sprint 6 was a built up from previous sprints. Also in our last meeting, we went over everything we did so far from sprint 1 through sprint 6 and it appeared a lot has been done though there is still some work to do.

Sprint 6 meetings went on successfully without any interference and we had very effective meetings and discussions. Among the activities that enforce effectiveness in sprint 6 was work sharing. Jason share the work among us and that kind of put everybody busy trying to come up with specific tasked result. We held effective discussions, with every member of the team coming up with some ideas which usually called for more explanations and discussion for consideration.  Rick Phillips continued to offer advice but this time, was very involved actively in trying to help us have something presentable at the end of the sprint. Jason also continually integrates ideas we came up with from our meeting, slack interaction with the team members especially Rick Phillips and from DJ of AMPATH into code.

Also, we moved away from encryption because we think other teams are working mainly on that aspect of the project and we doing that too seems double work to us. This idea helped us stayed focus on our offline data capture task.

Besides, from sprint 5, the issues we had with pouchdb was the fact that it capture a bunch of data which we think might have not been what been what we wanted; we at least wish to filter the result into certain fields in order to handle the limitations of storage space and that led us to trying to come up with a new GUI that will at least capture enough data for the user when he/she is offline. After several discussions, we could not exactly tell what field to keep and what to leave out. With regards, Rick Phan and Jason contacted DJ again as to what exactly they want for the new GUI. DJ then gave us a list of five things they wanted the new GUI to display. They wanted the user to b able to get the General info of a patient, Patient visits, Vitals, Lab results and HIV Summary. I was tasked to design a mock up for this, which I was working on. But DJ wanted to see it as soon as possible and Jason quickly responded to it and DJ accepted it and the team sticks that model.

In summary, I think sprint 6 was very successful and I love the enthusiasm and passion that we put into work. Even though, we are not totally done and the semester is almost ended, the team has something to contribute to AMPATH. I believe the work sharing that was listed on trello board also helped kept everyone busy. For the first time I learnt an abstract way of how to design GUI and mock ups in blasamiq in sprint 6.

From the blog CS@Worcester – Computer Science Exploration by ioplay and used with permission of the author. All other rights reserved by the author.

At the end of last sprint, we had finally resolved compilation problems that 3 of our 5 team members were experiencing, and we have Jason Knowles to thank for that.  Our team felt a rejuvination of morale at the beginning of this sprint, and we began finally resumed work on implementation of our offline encryption-service.  We split up the labor of implementing the service, among our 5 team members.  I was made responsible of implementing functionality for ‘salting’.  In encryption, a ‘salt’ is piece of random data that is used as an additional input to a hash function.  Within the hash function, the salt is concatenated to the password or passphrase to be hashed, which will help defend the encrypted data from rainbow table attacks.  I spent our first couple meetings researching salt, and was absent for most of the third meeting because I had to attend a lecture.  As I have mentioned in earlier blog posts, our team planned to use the crypto-js encryption library for our offline encryption service – but when I did make it to the third meeting, my team members were having problems importing the crypto-js library into the Ampath application source.  I had not experienced this problem yet because I had been doing all my implementation for the service within an npm runkit that had crypto-js integrated.  This problem, apparently, ruined productivity for that meeting and the majority of it and the following meeting were spent trying to resolve this issue.  Our teammate, Henry, began to have errors within his application toward the end of the sprint, haulting his progress.  We were unable to resolve these issues by the end of the sprint.

During the second half of the sprint, I worked on implementation of the salting functionality.  We discussed approaches with Matt Foley, of one of the other development teams, and we came up with a plan.  Their team is working on an offline login service, and login data is currently the primary focus of our offline encryption.  We decided that the salt for login data encryption would be the user’s username and the timestamp of their computer at the time of their login.  One meeting was spent merging changes from our group repository with my working repository.  Once that was done, I spent the remainder of the sprint writing a mock service for salting functionality.  I did my rough work in an npm runkit with crypto-js integrated, because we were not able to figure out how to import crypto-js into the Ampath application.  This is a major problem as we do not have any more time to research other encryption libraries.  We decided that we will just all write mock implementation for the purposes of our end-of-semester presentation.  The implementation for salting functionality seems pretty simple to me it is just incredibly unfortunate that we can not get the library imported into the application.  Professor Wurst said that we can still work on implementation if we wish, but we will have to reserve only a small amount of time for it as we have to prepare for our presentation.

From the blog CS@Worcester – by Ryan Marcelonis and used with permission of the author. All other rights reserved by the author.

This last sprint cycle for my team was probably the most productive sprint in terms of producing code but we ran into a lot of issues when it came to dealing with Promise objects. As I have mentioned in my previous blog posts, when making calls to to a pouchDB instance (i.e. db.put(), db.remove(), db.get(), etc…) those calls return what is known as a Promise object. At Mozilla.org I found a good definition of Promise objects. They describe them as “a proxy for a value not necessarily known when the promise is created.”  Hence, using promises allows developers to write asynchronous code fairly easily. The reason I say this is because asynchronous functions may have multiple expected outcomes/return values without knowing the order in which they will be generated. Before promises, asynchronous functions were handled using callbacks which would either deal with any errors the function might generate or execute the function as expected. You can refer to this link for a more in-depth explanation on the benefits of using Promise Objects instead of callbacks for asynchronous code.

Below is a diagram depicting how promises operate:

As you can see above, promise objects may be chained together using .then() which provides the capability to easily write functions with multiple asynchronous actions expected.

Initially, my team and I didn’t have a great grasp on the concept of Promise Objects and were expecting our code to return explicit values. Here’s an example of our addDoc() function in which we expected to get a boolean value to be returned:

The problem with this is that our function was actually returning  a promise object which that returned true when the necessary conditions were satisfied. After doing some research and working with the team who were writing the offline data capture service we realized our function was actually returning a Promise Object. Furthermore, we figured out that we could expand the functionality of our addDoc() method by utilizing the chaining of promises. With this knowledge, we re-wrote our addDoc() method so that it expected a Promise Object to be returned and could update an existing record if it’s ID was found in the PouchDB instance. Here is what the addDoc() function looked like after making those changes:

Testing functions that return promise objects also proved to be quite difficult. Here is a screen shot, taken from my gitHub page, depicting the test we wrote for the addDoc() method:

As you can see, highlighted in red, we initially had our function call inside the expect() part of our test. Then, we changed our test (shown highlighted in green) by moving the expect() inside of the .then() call after the addDoc() function. Subsequently, our test successfully passed and we thought we were on the right track. However, some of our functions had long chains of promises with multiple asynchronous actions expected. Thus, some of our tests were producing output that was hard to explain because we were still expecting our code to execute in a synchronous manner like if-else statements. In other words, anticipating that the result of each .then() call would be returned in a synchronous fashion, which was not the case.

In hindsight, a better method of testing our functions may have been just simply testing whether the correct pouchDB function was getting called the right time. We were trying to test whether the promises returned in our functions were, themselves, returning correct values. It may have been better practice to just log the responses of promise objects called in our code rather than testing the expected return values since asynchronous code may return things in an unforeseen order.

From the blog CS@Worcester – Caleb's Computer Science Blog by calebscomputerscienceblog and used with permission of the author. All other rights reserved by the author.