When It’s Easier to Just Do Everything [More] Manually

Sometimes doing things the hard way is a lot easier. The more tools you use and the more complicated those tools are, the more complexity you have to deal with. So while it may be nice to call a few simple methods and have a framework do everything for you behind the scenes, you’ll have to to learn how the framework works and maybe realize down the line it can’t do everything you want it to do. There may even be incompatibilities with other parts of your program.

This week in my independent study, I tried to figure out how I could run a machine learning model on Android. I had some success, but quickly discovered some complications. Android has the option of using TensorFlow Lite, which seems great. However, I built my model using Keras, so I needed to convert the model. That was relatively straightforward, but before I started calling my model, I realized that I needed to extract audio features on Android. This required using Python code on Android, particularly Librosa and Numpy. This led me to other potential frameworks to get this to run.

This would lead to a bloated app, so I looked into Google Cloud services and thought about running server-side code there. I already set up a way to upload and download files with Google FireBase, so this seemed reasonable. But this is a paid service and would be even more work to make it functional.

I already have all the code running on my personal machine, so what if I just set up a server with a REST API to upload and download files and run the necessary Python code locally? If I could get that working, it would be trivial to call the code I’m already running.

Getting the server to upload and download files is what I did this week. I used Flask, which makes it very easy to get a basic server up and running. For the time being, data can only be transmitted via WiFi, as there will be uncompressed audio files transmitted back and forth.

While there was some additional work to figure out HTTP requests on Android, already knowing the basic building blocks gives me much more flexibility moving forward. But with great flexibility comes great responsibility, and proper error checking will be an important part of development moving forward. Security measures are also very important to consider before deploying an app to production.

The next iteration will involve running the machine learning code with a REST API call and getting back both the results of the model’s prediction and any data I will need to plot within the app.

From the blog CS@Worcester – Inquiries and Queries by James Young and used with permission of the author. All other rights reserved by the author.

Apprenticeship Patterns: Breakable Toys

The second Apprenticeship pattern I would like to discuss is titled “Breakable Toys.” This pattern is based on the idea that experience is built upon failure. Because many software developers work in an environment that does not allow them to fail, they have trouble learning more about development and improving their skills. This pattern encourages developers to work on smaller projects outside of their work environment, giving them a way to experiment and make mistakes without the usual consequences. Creating these “Breakable Toys” allows developers to learn new things about their usual tools while working on enjoyable projects, which helps them gain useful experience that they could not have acquired in their usual work environment.

I chose to share this pattern because of my own experiences working with Breakable Toys. When I was first learning to program in high school, I would always spend time outside of class working on my own projects. My passion for programming was at its height when it was still new to me, as I constantly wanted to learn more about it. Since starting college, however, I have felt that passion gradually fade. Programming became a central focus of my education, especially after transferring to WSU for Computer Science. My fear of failing in my classes caused me to associate programming with stress and anxiety instead of passion and enjoyment. I eventually stopped working on Breakable Toys, as my attention was completely directed towards classwork and I felt that personal projects were a waste of my time.

Reading this pattern has helped me realize how important my Breakable Toys really were. They contributed to my learning far more than any classwork ever has. They allowed me to figure out new skills at my own pace and let me experiment with them without worrying about failure. For example, due to my focus on Game Development, my Toys helped me learn more about programming graphics than my classes have and they helped me understand the basics of how game engines function by programming them myself. This pattern has made me realize that my personal projects were never a waste of time, as they helped me learn new skills quickly and enjoy myself while doing it. Going forward, I hope to get back to working on Breakable Toys outside of my work environment, as I now realize how important they are to my learning.

From the blog CS@Worcester – Computer Science with Kyle Q by kylequad and used with permission of the author. All other rights reserved by the author.

Retrospective Sprint-2

This issue was to create a Backend Rest Api that could communicate with both our database and Front End UI.
https://gitlab.com/LibreFoodPantry/modules/visitmodule-tp/checkout/Plan/-/issues/11

This issue was done early in the sprint to see if MongoDb was a viable option to use for our project.
https://gitlab.com/LibreFoodPantry/modules/visitmodule-tp/checkout/Plan/-/issues/8

This issue was to create a definition of done for our issues so that the team had a better structure to follow when working.
https://gitlab.com/LibreFoodPantry/modules/visitmodule-tp/checkout/Plan/-/issues/23

In sprint two our team excelled in taking on the correct work load. Sprint one left us with too little work and we had to add things while we were working and even did some work without creating issues. However, in sprint two we proposed our work with proper weight classification and so we had just enough work. Our workload reflected our Sprint length well and we were able to split up the workload well between team members. This time around we took large tasks that had several parts and instead of weighting them higher we broke the issue down into several smaller issues. This way we were able to spread the workload out throughout the team and didn’t leave anyone working on one large portion of work for too long. I think this worked to our advantage because it was easier to formulate a plan and track our progress this way. As we completed more smaller issues we could see how far along we were with progress and dictate work to team members better. 

We didn’t have much of a problem at the beginning of the Sprint, although, towards the end when school was moved to online classes only we began to have some issues. It may not be within our power to control but the move to online classes was a detriment to our work ethic and communication as a team. Having a second week of spring break that otherwise would have been a week of working was a blow to motivation and the team’s communication was lacking. Now that we have returned to classes online we have gathered ourselves well and are ready to work. However, the removal from our work for twice as long as we had originally planned deemed to be a detriment to our momentum. We still completed the work load we had planned but we felt rather scattered.

Moving forward we need to work on our communication as a team. Before the national crisis we would meet in person as a group weekly and work for three or more hours together. This synchronous work ethic proved to be our most productive environment. Now that we cannot meet in person I think we could plan for synchronous online meetings to help aid our disconnect. Even if we are not collaborating on issues I believe it could help morale and productivity to know we are all working simultaneously.

As an individual I believe I just need to get back in the groove. Hopefully, with the help of my team we can create a schedule to add some structure to our workflow and get back into a forward momentum.

From the blog cs@worcester – Zac's Blog by zloureiro and used with permission of the author. All other rights reserved by the author.

Apprenticeship Pattern Review #7:Dig Deeper

In practice, algorithm problems do not arise at thebeginning of a large project. Rather, they typically ariseas sub-problems when it suddenly becomes clear that theprogrammer does not know how to proceed or that thecurrent program is inadequate.Steven S. Skiena, The Algorithm Design Manual

From the blog CS@Worcester – Shams's Bits and Bytes by Shams Al Farees and used with permission of the author. All other rights reserved by the author.

LibreFoodPantry Sprint 2 Retrospective: Controllable and Uncontrollable Variables

Here is what I worked on during this sprint:

Sprint Reflection:

It has been so long since the onset of this sprint that it is somewhat hard to remember how it even started. What I do remember is that coming into this sprint, my team and I were determined to get more work done than in the previous sprint. It looked like it would have been relatively easy since we had a longer sprint this time and our collaboration had improved greatly since the start of sprint 1. The start of the sprint proceeded smoothly and I took some intro to UI design courses to get the mock UI done within the first couple days of the sprint. We acknowledged that more work had been proposed for this sprint, but we had more time to get it done and I was confident that I would have the time over spring break to get both frontends completed. I think up to the point where all residents on campus had to move out because of the danger of COVID-19, I think my team’s communication between each other and planning was much better than the previous sprint. Issues on our board were much more granular and detailed and all of them were linked to the proper Epics and had the proper tags. I think all members of the team started to take the Scrum practices more seriously and it showed in our organization and communication.

This is when all of the uncontrollable variables of life hit us like a truck. All of the stress and changes in dealing with a worldwide pandemic definitely took a toll on my productivity. Rather than spend this whole retrospective talking about how COVID-19 ruined my plans for getting the frontends completed, I’ll talk about how we could have better prepared for this situation. Spring break was extended by two weeks and there was nearly a month without communication between our team. Over this time I felt uncomfortable asking for help or clarification about issues I was having with my team since I had no idea what anyone was going through. I decided it would be best to hold off on development until classes started up to give all members of the team time to adapt to the fully remote life that we are now living. While this is an extreme case, I think this is a good lesson in accurately weighing tasks as well as not adding too much work to a sprint. While creating issues I think my team and I fell victim to weighing everything using a “best-case scenario” approach. All of the weights created reflected the time it would take for us to complete a task if nearly nothing went wrong. As a software developer, I should have known that failure is expected and every task will almost always take longer than you think. Next time weights will reflect the reality that things go wrong so we can more accurately fill up our sprint with tasks. When determining what we would put in the sprint we were taking into account the time we’d have over spring break. This meant to get all of the work done for the sprint we’d have to put in a good amount of effort over the break. The misjudging of weights combined with this meant that an event as serious as coronavirus caused us to miss our sprint deadline for a lot of the frontend work.

Going Forward:

As an individual, I will be more responsible and truthful when weighing tasks. I will also communicate with my team and other teams as soon as problems arise so we can resolve any issues I am having as soon as possible. As a team, I think the place we could improve the most in is reviewing completed work. Allowing merge requests to pile up on GitLab causes development to slow down until those get merged. We decided to try and keep our “Needs Review” column to two issues at all times, but this was not enforced as much as it should be.

From the blog CS@Worcester – Creative Coding by John Pacheco and used with permission of the author. All other rights reserved by the author.

Craft Over Art

I have endured the struggle between craft and art, beauty and functionality. The dream for any project is to master both but most times it is unrealistic that you will have the time to fully master either one without needing maintenance. Since there is no guarantee of the complete fulfillment of either functionality or beauty you must find a common ground. This is not a common ground of what you think should be but a common ground to make the customer or user happy. You can’t go all functionality and present a fugly work, even if it an amazing work of code. Vice versa you cannot pass in a beautiful shell of a Web User Interface with feature that don’t work correctly!

My favorite part of this chapter was the separation between a craft or art and fine art. It’s not about your own expression, it’s about meeting the threshold of what will make your customer happy and developing past that if possible. First you need to create a great but simple work that meets the basic needs and looks presentable, then you may chase functionality or beauty.

Another great point in this reading was that the best way to learn can be fixing mistakes rather than starting over just because you messed up. If you go to far with functionality or beauty and one starts to fail because of the other than you need to stop and find out why. You may learn something that will help you in the future, hone your skills. Refactoring and repairing may be more beneficial for you and the customer.

I know I have experienced this issue in a slightly different scenario. My partner and I were creating a Web User Interface that connected to a database and we had both come up with basic models for the code. Instead of starting a new project from scratch we refactored one of our projects to incorporate the things that worked in the others program. We learned many lessons on functionality and beauty by fixing code rather than restarting. Since our code was a mix up of each others design we had to find out how our web page would change. We ended up going with the best functionality we could get with a little less beauty than we liked. However, it worked great and was up to standards with it’s appearance so the result was a success.

From the blog cs@worcester – Zac's Blog by zloureiro and used with permission of the author. All other rights reserved by the author.

Apprenticeship Pattern Review #6: Practice, Practice, Practice

The people we know as masters don’t devote themselvesto their particular skill just to get better at it. The truth is,they love to practice—and because of this they do getbetter. And then to complete the circle, the better they get the more they enjoy performing the basic moves over andover again.—George Leonard, Mastery Each one … Continue reading Apprenticeship Pattern Review #6: Practice, Practice, Practice →

From the blog CS@Worcester – Shams's Bits and Bytes by Shams Al Farees and used with permission of the author. All other rights reserved by the author.

Dig Deeper

For my final blog post on apprenticeship patterns, I wanted to discuss my favorite pattern. Software is so pervasive now that anyone can make a working product with little more than superficial knowledge of a language and a framework. This is great motivation to continue, but it may lead one to erroneously believe they are an expert. Finishing a product, even a successful one, doesn’t make you an expert programmer.

Digging deeper is going below surface knowledge of a technology and learning the nitty-gritty bit-y details. The caveat is to not become too specialized. The book warns to keep your perspective of the project as a whole, and only the learn as much detail necessary to help with a given task or problem.

I was originally taught to treat new classes as a black box, and I only found it frustrating once I graduated to more complicated tools. To truly understand how something is meant to work, you have to look inside. Another example: I’ve taken a few introductory classes that used metaphors to explain concepts and/or taught from the top down, adding detail over time. Biology class was boring and difficult because I had to memorize that a blue circle will separate the green, spiked lines so that two red hexagons can copy each of them. It wasn’t until high school, which provided an understanding of underlying chemical reactions, that biology became interesting and easy to remember.

So it is with software. I’ve been exceedingly frustrated with new tools when I tried to play without understanding. Sometimes, it works. Others, when things begin to get confusing, diving in becomes a necessity. Another caveat: you don’t know what you don’t know, and if you assume you’re doing it right, you may be wrong. Even if it works.

This is another pattern that requires balance. Learning details provides diminishing returns over time, but you should mostly understand why you need to do something a certain way, and how it is working. If you can explain this in simple words, you’re probably on the right track. This applies not only to software tools, but work processes as well.

You may not always agree with how a technology was designed. No one will tell you that the modern Internet is a perfect design, because it has been manipulated into working in a world it wasn’t designed for. Created in a world of text, it now works in a world of streaming video across billions of devices. This would never have been achieved without engineers and developers who understood the basic building blocks of the technology. Be like them.

From the blog CS@Worcester – Inquiries and Queries by James Young and used with permission of the author. All other rights reserved by the author.

Improving the Spoken Digit Speech Recognition Machine Learning Model

After getting a simple machine learning model to recognize spoken digits, I was able to begin the iterative process of improving the model. Using only MFCC’s, the model was failing more than desirable, reaching a maximum of 60% accuracy when using validation data (my own voice, which was not used in training the model).

Below you will see plots of a sample of results when validating the model. For each digit, there is the extracted MFCC features, the actual spoken digit, the predicted digit by the model, and the certainty. There is also a plot of the certainty for each other digit for that recording.

This is just a sample of a larger validation set, and the actual results in this first model was only 45% accurate. But this shows that for all of these digits except 3 and 5, the model was 99% to 100% certain of the result. The differences in the MFCCs are subtle, but stark differences in color appear to be more likely to be correct, whereas 5 is clearly closer in color to 1, which it was mistake for. Additionally, every single audio clip of 3 was mistaken for a 0 using this model.

Retraining the model with different parameters may help in this case, but we can also hypothesize about the reason for these mistakes. Perhaps the MFCC is finding patterns in vowels that make “zero” and “three” look identical. If that’s the case, features that can detect consonants might help improve results. This sounds pretty obvious anyway, so it might be a good next step on the next iteration.

But first, let’s retrain the model without any changes.

Okay! This 3 was very accurately predicted. But the total accuracy of validation was only 50% (remember, this only shows a sample size of 10). Inspection of actual results now shows that 3 is sometimes mistake for a 2, and vice versa. This model is slightly better, bit still flawed. Which makes sense, because no changes have been made to the model and we just got lucky that it learned to be a bit better this time.

I’ve been training with 25 epochs, and getting 95-97% accuracy during training, and 93-97% accuracy using test data (from the same dataset as the training data, which was not used to train the model). Those results are pretty good, so maybe we can use fewer epochs and prevent some overfitting.

This certainly looks promising. With 95% accuracy during training, and 93.8% accuracy using test data, the results are still pretty good. However, the validation data with my voice is now 57.5% accurate! Only a single 3 was mistaken for a 0.

So I’m using a dataset of 4 voices to train and test, and my own voice to validate. But more data is probably better, so let’s use my voice to train the model and take a random sample to validate.

The plot is looking good! Each of these was very accurately predicted. During training and test data was 97% accurate. The validation data was 100% accurate. Of course, now that the validation data contains all voices that were trained, it’s more likely to be correct. Furthermore, the sample is small. So let’s see what happens if we use a new voice to validate. I had my roommate record himself saying each digit and used only his voice for validation data.

In general, the model is much more certain of its guesses. The final validation result was 80% accuracy, so not perfect but a major improvement. This much of an improvement was gotten just by adding more data and making small modifications to the model.

The importance of collecting data in order to improve a model is apparent. Even with 80% accuracy, there is still some predictive power. If this can be found to be useful, further data can be collected as it is used and this new data can be cleaned and used to train better models.

From the blog CS@Worcester – Inquiries and Queries by James Young and used with permission of the author. All other rights reserved by the author.

Sprint 1 – Retrospective

Sprint 1: 01/22/2020 – 02/19/2020

During the month, I started to work as part of the UpdateGuest time for the LibreFoodPantry project. The Sprint started with sharing and learning general knowledge about the project, what are the services to be offered, and some future perspective. As I learned the service that I and my team will be supporting, we began to fill the backlog with to-do tasks. We operated as a Scrum team, managing a Scrum board in GitLab, and separating stories based on requirements.

We started this sprint by learning new knowledge. As we decided to with the MEAN Stack, we needed to gain knowledge in all technologies that will be used: MongoDB, Express Framework, Angular, and Node.js. So we created stories to track our learning timeframe and weight the knowledge we gained.

During the Sprint I worked in multiple stories and supported the team in different topics:

  • #1– I created the Angular App, the Mock version of our UI service. I supported the team with discussions on how the App should be implemented and organized.
  • #2 – I supported to edit and fix an issue we had with .gitignore file. Together with the team, we discussed how to fix this issue and concluded with a good result.
  • #3 – I created the Document and designed the Frontend Architecture for out UpdateGuest. This document will support the implementation of the service. This document will be edited during Sprint 2
  • I supported the team with PR reviews, issue discussions, and research when we had knowledge gaps.

The end of the first Sprint consists of secure knowledge and the beginning of work with UpdateGuest service.

In this retrospective, I and my team need to reflect on these points:

  1. What we need to stop doing
  2. What we need to do less
  3. What we need to keep doing
  4. What we need to improve

1- What we need to stop doing

We should stop Approving PR by only one person. Each member of the team should approve a PR – so we know that we are all on the same page.

2- What we need to do less

We should be careful when we create new stories and check if there are no duplicates.

If there are duplicates, before closing the story we should let the creator of the story know first.

In the case of merging a PR, the creator of the PR should merge, or if another person do it, we should let the creator know first.

3- What we need to keep doing

We need to keep working with stories that we have on the Sprint backlog.

We need to discuss issues that we may have during the sprint and approve each other work.

We should keep reviewing each story before we mark them as DOne and close them.

We should keep solving debatable conversations and conclude in a good result that works for everyone.

4- What we need to improve 

We should have better communication for each story, issue or blocker during the sprint. Class meetings are not enough to solve issues. For each story, we all need to discuss and communicate with each other, and to consider all opinions before we conclude to a solution. Communication is the key to success for a team.

Conclusion

As a team, we successfully closed Sprint 1. We closed most of the stories in Sprint’s backlog and supported each other to move to the next sprint. We had a Sprint review for what we did, and a Sprint planning for Sprint 2, so we are prepared for the next step. So, let’s start implementing our ideas!

From the blog CS@Worcester – Gloris's Blog by Gloris Pina and used with permission of the author. All other rights reserved by the author.