Category Archives: Week 5

Audio Feature Extraction

This week I’ve continued working on the application part of my Android application, but I’ve also started to “dig deeper” (apprenticeship pattern blog post coming soon) and learn a bit more about audio processing.

I started with general concepts of feature extraction a couple of months ago now. Understanding how to use Python libraries to extract features is simple enough, but actually understanding how they work and why they work is another story. This week’s research has revealed an underlying elegance to the concept of signal processing and helped me reach a higher level of understanding and excitement for this project.

The single best explanation has been a video by Youtuber 3Blue1Brown on Fourier Series. I recommend all of his channel because he has an elegant way of describing and visualizing every topic he speaks about. He helped me understand the beauty of Calculus, and in my process of digging deeper I wound up watching his video on the uncertainty principle which was surprisingly relevant to signal processing. Understanding the specifics of the math behind signals and waves, and knowing the fact that mathematical equations are a language used to describe straightforward physical phenomena is key. This knowledge makes daunting concepts easier to break down. Seeing the same concepts used in different contexts also helps solidify them in your mind. And if you’re implementing this in code, it will make it much easier to remember the necessary logical steps required to extract a feature.

This entire tangent (and however useful, it was an unexpected tangent) started with trying to better understand the types of feature extraction that are used in speech recognition. By the way, you know you’re digging deeper when an article with an estimated read time of 11 minutes takes you a few hours to get through with all the additional research.

And universally, as far as I can tell, the first step in signal processing and feature extraction is the Fourier transform, which is simply turning a raw audio signal into separate sine and cosine signals. I say simply, but as the 3Blue1Brown video states, this seems a bit like figuring out which colors make up a mixed up can of paint. It turns out, however, that clever math makes it quite obvious which summation of sine and cosine signals make up a complex signal. I encourage you to watch the video to understand why.

The summation of cosine and sine waves is considered the frequency domain, while the original signal is in the time domain. From the resulting frequency domain, the individual signals can be normalized by taking the log magnitude of the signals and performing an inverse Fourier transform.

This is a new concept called a cepstrum, and it is one of many possible transformations you can make on a signal to begin to analyze the data. Its usefulness comes from the ability to see changes in individual waves. Additional operations can be performed to reveal new insights into patterns in a signal. Determining which if these works best is part of the process.

These individual transformations would be very interesting to implement in code. I may not get a chance to do so for this project, but the understanding of the underlying operations will help in using existing libraries.

From the blog CS@Worcester – Inquiries and Queries by James Young and used with permission of the author. All other rights reserved by the author.

Breakable Toys

For this week I’ll be covering the pattern, Breakable Toys, from Apprenticeship Patterns: Guidance for the Aspiring Software Craftsman. The breakable toys pattern uses small personal projects to teach new techniques. When in a professional environment, playing and learning with the project is not an option. To keep ourselves learning new skills after college, the best thing to do is to create personal projects that allow us to make mistakes and grow as developers. These projects, or breakable toys, are chosen by us and will likely hold more interest than work projects. By working on these projects, we can learn the workings of a project without the fear of consequences of our mistakes.

This pattern is a bit obvious, but I still think it is worth reminding ourselves of. I sometimes find myself tired of working on a needed project and begin to lose some enthusiasm. A breakable toy is always a fun and educational way to lift my spirits while still progressing my skills. To me, breakable toys are more than just a consequence free environment to develop in, breakable toys renew my enthusiasm in the craft.

When I just started learning programming back in ’08, I felt like a whole new world had opened up. There was so much to see and learn, and I was psyched for what I was to learn. My first breakable toy was a text-based game. It was terrible, messy, and bugged but I learned a great deal in the process and because the project was personal, I could do whatever I pleased without concern. While I have not worked with that project in over a decade, I can still remember some of the lessons I first learned while making that game.

I would highly recommend exercising this apprenticeship pattern. This pattern combines the developer’s interests with a stress-free environment to create a project that is exciting and rewarding. I think of our coding skills like cooking, part science and part art. To grow as a programmer, it is important to remember that we need to push our boundaries and experiment is we are to learn. A breakable toy is the perfect project to practice old skills and develop new ones.

From the blog CS@Worcester – D’s Comp Sci Blog by dlivengood and used with permission of the author. All other rights reserved by the author.

Capstone Sprint 1 Retrospective

With our first sprint under our belts, our team is excited to dive into more advanced features. A majority of this sprint was spent learning some new technology, getting our heads around the project’s documentation and workflow, as well as learning to work as a team. However, we also made some strides in implementing our final product.

My contributions

Discussion about whether an issue was a duplicate or if it was a typo, and subsequent editing of the issue.

Commenting with documentation for date format for future reference, which occurred during an in-person discussion.

Request to take care of an issue that was already assigned, in an attempt to complete the tasks at the top of the board first.

Opening an issue that was discovered while committing with a new .gitignore file.

Reviewing and marking an issue to find approved colors and logos as done after completion.

Implementing a stub and merging after approval.

Discussing issues during approval of another feature and merging manually.

Retrospective

As a team and individuals, we are all excited about this project. We all bring our own interests, skills and knowledge, which come with our own quirks and blind spots. This occurs on any team, but it is nice to acknowledge this in ourselves and in our teammates so that we can become more willing to spot them in each other. This allows us to delegate work that we need, ask for help when we need it, and call each other out when we are going down a rabbit hole that is a dead end. I may be the worst offender in this, because I would sometimes slow myself down getting caught up in unnecessary details. I wouldn’t resent my team for telling me I’m worrying too much. I personally feel I could also be better at describing what I’m worrying about.

This experience during this sprint opened up a possibility for improvement for our next sprint planning: more detailed descriptions of “done”. Being new to the format, we tried to shoehorn our issues into a standard template that we were provided. Using only the “given, when, then” Gherkin format, we didn’t get a chance to fully express what we wanted done. Elaborating more in the initial issue will help our focus and prevent worrying about adjacent, unrelated issues, solely due to thinking about each problem in more detail. For example, our stubs didn’t include any testing as part of the definition of done, but luckily we felt that this was reasonable and included them, rather than creating a new issue. I am of the opinion that for software to be complete, some testing should be associated with it. We will create a nightmare down the line if we aren’t careful with this, and should be more clear when defining our issues.

Much of our work was also done as a team, together in the same room. This forced us to remember to document our discussions and decisions in GitLab after the fact. We could have been more diligent in this regard. Although many of the solutions seemed obvious to us, documentation on our reasoning could be important for future developers. I enjoy working in person with a team because it facilitates quick discussion, but I feel we should either discuss more over GitLab or hold each other accountable for documentation after a decision is made as a team.

We are all feeling pressure to finish something we are proud of by the end of the semester. I think this contributed to some of the worrying and long discussions we had on features.

Making matters worse, was that other teams were likely having the same issues we were in adjusting to the new project. I think we handled this rather well and had some good back-and-forth about API design and the features we were planning on finishing. Again, this could have been better documented on GitLab for future reference. Some of this occurred in Discord, and will be lost in a sea of other messages. The less-obvious decisions were luckily documented in GitLab, such as the design of the ApproveGuest Module API. I would have liked to see more back-and-forth between our two groups in the discussion, instead of an abrupt approval of the feature.

This was a short sprint with an emphasis on learning, so thankfully we will be able to improve on these issues and focus on our strengths in the next longer sprint, which should get us close to a working product.

From the blog CS@Worcester – Inquiries and Queries by James Young and used with permission of the author. All other rights reserved by the author.

Draw Your Own Map

This week, I’ve been thinking about an apprenticeship pattern that brought up something I had never considered. It opens with a quote suggesting people might say programming might not be sustainable in the long-term. News to me!

The crux of this pattern is that you won’t always be provided with the career path that you want. To be successful, you have to fight the urge to follow the path an employer gives you and take steps toward the path that you want. This is your map, and it can be redrawn at will. The goal is not to narrow your experience just because of a comfortable salary and your attachment to a fancier title.

This pattern seems to stem from the loss of many great developers to the world of management. The author seems a bit biased in wanting to prevent this. Unfortunately, many people might find their salaries necessary to maintain a lifestyle, or support a family. Furthermore specialization, or “narrowing” as they describe it, can be a good thing.

Of course, if narrowing is part of your map, I suppose it still fits with the pattern. But the book also suggests looking at the possibilities, which I agree with. After all, a map doesn’t have one path; it has many from which to choose. Listing many potential paths and following the one that looks best for you is likely a good tactic to get where you’d like to be in your career.

I’ve always felt a pressure to choose a single path and stick with it. Being able to change my mind is a nice idea. Luckily as far as I know now, I’ve mostly figured out my career path, but I will also follow the suggestion of the book to plan many alternatives. Experience in multiple different positions has given me insight into the aspects of each job I like. With this knowledge, I’m only interviewing at positions that I trust I can stay with long term, and I’ll only accept a position if I still believe that’s the case. Nevertheless, the rest of my career is a long time, and companies change.

The quote at the beginning of the chapter is meant to say that if people tell you to stick to one thing — to move to the safe, normal career path — you don’t have to. If a company prevents your own professional growth, it’s time to jump ship.

From the blog CS@Worcester – Inquiries and Queries by James Young and used with permission of the author. All other rights reserved by the author.

Learn How You Fail

Ingenuity is often misunderstood. It is not a matter of superior intelligence but of character. It demands more than anything a willingness to recognize failure, to not paper over the cracks, and to change. It arises from deliberate, even obsessive, reflection on failure and constant searching for new solutions.

-Atul Gawande, Better

Learn how you fail is the first step towards learning how to win. The objective is to gain self-awareness of the reasons which lead to failure, and then make better decisions by reducing the inclination towards idealism. Being conscious that you cannot know everything, is the only way to become aware of how you fail. Knowing your own limitations, allows you to deliberately identify obstacles and concentrate on the goals you can reach.

Yes, you will fail! What’s more important is that you will win, once you know why you failed. The way to get the best out of it is to create a system to prevent the failure from happening again after you have figured out what went wrong. Creating a checklist or a type of protocol to prevent it from happening again when you find the weak point is another option to not go through failure again.

In this pattern what got my attention is the last sentence, which says “if you’re brave enough, get a friend to review your code, and see what else she can discover”. This is very accurate because everyone who writes code knows that there will be errors and it will take time to fix them. Even though you fix the errors you get, someone will find something else for you to fix, especially when you work in groups. This can be very tough because you feel like you are falling down. The reason why this happens is that we feed our brains to always win, and never lose. We reach the goal because we know how to do everything, and there are times when we do not reach the goal because there is something, we had no clue about. Is it time to step back? No! We should go through the failure, learn why you fail, and try again if you think you’re capable to fix it. As a conclusion, it can be very painful to learn how you fail, but in the end, you will have the skill to accept whatever life throws your way.

From the blog CS@Worcester – Gloris's Blog by Gloris Pina and used with permission of the author. All other rights reserved by the author.

The Deep End: Taking off my floaters

The definition of many of the apprenticeship patterns in the book are all in their title, being fairly obvious or sharing language used in idioms or other expressions. However, despite this, our authors usually find a way to expand on them for at least a page or two – most often by sharing a relevant story. Which is why I was shocked at how brief this particular pattern was, just a handful of paragraphs and another for our story. The brevity of pattern however did not lessen its impact on me, and I have come away with a renewed confidence. As someone who needs structure in order to work effectively, I imagine the deep end would involve a great deal of work and specifically responsibility – which I respond to quite well also.

I had already determined to seek out an internship or entry level position outside of my home state. Further, I wanted to join a company where I, of course, could be actually hired but one that would truly test the limits of my skills. Ideally, I would want it to be an internship though, cause as much as I want to make this change in scenery, I’m still hesitant to get stuck someplace I’m miserable. I think the true test will not just be getting a position away from home but finding a place for myself in that new place and maybe even gaining the confidence to stay and accept a job at the company – in the case of an internship.

I did find one internship that states that they will make no “intern-work” and one should be expected to work as a peer with the rest of their fellow employees; and on a massively used and well-known mobile application no less. I jumped at the chance to apply for this position, and I am incredibly hopeful I get it, as it is the perfect storm of conditions I am hoping for. Enrique’s story is still a little too dramatic for me to imagine emulating, but I hope if the opportunity to work in another country even presents itself, I will have the confidence to take it.

From the blog CS@Worcester – Press Here for Worms by wurmpress and used with permission of the author. All other rights reserved by the author.

Apprenticeship Pattern Review 4: Retreat into Competence

When a software developer gets overwhelmed by the complexity of their current project or the fact that there is so much they don’t know, it is okay to “retreat into competence”. In the face of a challenge that seems insurmountable, taking a step back to see how for you’ve come and what you have accomplished is okay. While retreating into competence can be the right prescription for times like these, it is important to use the traction gained while retreating to propel yourself beyond where you first had stopped.

The title of this pattern almost seems contradictory to the whole concept of becoming a software craftsman. A software craftsman should always seek to move forward in their knowledge and career, why would the authors be telling us to retreat? While being able to continuously march forward indefinitely, steadily increasing project complexity and your responsibilities would be ideal, it almost never works like this in the real world. I find this pattern to be quite important, because without it, it seems that one would be going directly against the principles of a software craftsman to step back from a problem and retreat into competence.

One thing to note is I feel like it is especially easy to fall back on this pattern too much. If you consistently find yourself pressing the “eject” button whenever you reach a problem you might not know how to solve yet how will you ever go forward. It is important to set constraints on how long you can retreat in order to protect yourself from truly going against the principles of software craftsmanship and living a career filled with mediocrity. This is a powerful pattern, use it responsibly.

I feel like this pattern can be used in many situations in order to get over hurdles in one’s career that seem too great. Allowing yourself to create things you feel comfortable developing can help you regain faith in yourself and software development as a career. This is a pattern I have used myself and have seen great success with. When I find myself lost or over my head with a given problem, I like to solve small programming challenges, sometimes referred to as kata, in order to instill confidence in myself and remind myself how much I have improved over my career.

 

From the blog CS@Worcester – Your Friendly Neighborhood Programming Blog by John Pacheco and used with permission of the author. All other rights reserved by the author.

Breaking Toys

The Breakable Toys pattern talks about a familiar lesson in life, which is that you can learn from your mistakes and failures because they are bound to happen. The pattern suggests that you create something similar to what you are doing in your job/work and use it as a playground to see what will work and what won’t and expand on the things that interest you and learn from what you create or fail to create. The one thing they suggest making and that a lot of developers make is a wiki because they simple to make and expand upon over time to show what you have learned and what you are capable of.
This pattern seems very useful and something that I might implement very early into my career because it sounds fun and interesting. The ability to create something and build on is something that is very important especially in the world of computer science where new things are always being made/optimized. I think this pattern makes me feel like the ability to grow and work on what interests you is an extremely important quality to have in software development. I now understand why our professors wanted us to make a professional blog and use it later in our professional careers since it will demonstrate our interests and progress made in learning more about the different aspects of computer science and software development.
The one thing that I really like about this pattern is the creativity aspect because it makes learning more fun since you are adding and making things more refined because you want what you create to look or work better. The best projects, not only in computer science courses, were the ones that allowed us to create whatever we wanted and pursue what we were interested in. Treating a job like this can turn it from a boring and exhausting job to a hobby that you enjoy and get paid to do. I really like this pattern and wish I knew about it earlier because I most likely applied it to my computer science courses as well to make them easier and more fun.

From the blog CS@Worcester – Tyler Quist’s CS Blog by Tyler Quist and used with permission of the author. All other rights reserved by the author.

Docker and Automated Testing

Last week, in my post about CI/CD, I brought up Docker. Docker can be used to create an “image”, which is a series of layers that are built upon each other. For example, you can create an image of the Ubuntu operating system. From there, you can define your own image with Python pre-installed as a second layer. When you run this image, you create a “container”. This is isolated and has everything installed already so that you, or anyone else on your development team, can use the image and know reliably that it has all necessary dependencies and is consistent.

Of course, Docker will get much more complicated and images will tend to have many more layers. In projects that run on various platforms, you will also have images that are built differently for different versions.

So how does this apply to CI/CD? Docker images can be used to run your pipeline, build your software, and run your tests.

The company released a Webinar discussing how CI/CD can be integrated with Docker. They discuss the three-step process of developing with Docker and GitLab: Build, Ship, Run. These are the stages they use in the .gitlab-ci.yml file, but remember you can define other intermediate stages if needed. The power of CI/CD and Docker is apparent, because “from a developer’s perspective, all you have to do is a ‘git push’ — and that’s it”. The developer needs to write the code and upload to version control and the rest is automated, with the exception being human testers who give feedback on the deployed product. However, good test coverage should prevent most issues and these tests are more about overall experience.

Docker CI and Delivery Workflow
From Docker Demo Webinar, 4:49

Only five lines of added code in .gitlab-ci.yml are necessary to automate the entire process, although the Docker file contains much more detail about which containers to make. The Docker file defines the created images and the code that needs to be run. In the demo, their latest Ubuntu image is pulled from a server to create a container, on which the code will be run. Then variables are defined and Git is automated to pull source code from the GitLab repository within this container.

Then, a second container is created from an image with Python pre-installed. This container is automated to copy the code from a directory in the first container, explained above. Next, dependencies are automatically installed for Flask, and Flask is run to host the actual code that was copied from the first image.

This defines the blueprint for what to be done when changes are uploaded to GitLab. When the code is pushed, each stage in the pipeline from the .gitlab-ci.yml file is run, each stage passes, and the result is a simple web application already hosted from the Docker image. Everything is done.

In the demo, as should usually be done in practice, this was done on a development branch. Once the features are complete, they can be merged with the master branch and deployed to actual users. And again, from the developer’s perspective, this is done with a simple ‘git push’.

From the blog CS@Worcester – Inquiries and Queries by ausausdauer and used with permission of the author. All other rights reserved by the author.

Exploring the Singleton Design Pattern

As we start to learn about more design patterns and their implementations in class, I wonder more and more about their application in the real world, as well as how they are perceived by modern developers. So far, for each design pattern we have learned, a relatively good job has been done in showing the advantages and pitfalls associated with them.

I became particularly interested in the Singleton pattern after finding the blog post: “Understanding What Singleton Pattern Costs You.” found here: https://blog.ndepend.com/singleton-pattern-costs/. There are also other sources I found that seemed to really dislike the Singleton design pattern, and I wanted to delve deeper into why that is.

In our in-class example with ducks, the Singleton solution seemed to make some sense for what we were trying to do. Rather than creating a new object every time we wanted our ducks to fly, we could have a single object and change its instance when needed. Because of the limited scope of what we were doing with the singleton pattern, we didn’t get a clear example of this practice potentially going south.

One way this can cause problems is that it can obfuscate code. Because a singleton is now essentially “global” and can be called everywhere, certain methods can do things that are part of this singleton that the method itself might not even indicate that it does. An example that the author of the blog provides is this:

In this example, the method is named BuildSimpleOrder, however, this method also logs the order. This makes it so you can make no assumptions about what your methods do, as many of them may have hidden behaviors. This has lead to a phrase that I have seen around tech blogs saying “Singletons are liars.”

This also causes problems in unit testing for similar reasons. Because you can use these singletons essentially globally, it can make it a lot harder to track problems in individual classes. This leads to high coupling. This also breaks the single responsibility principle – something that has been repeated in class a few times.

Learning about the different design approaches that we can take, with implementations evolving as the requirements change has been a good learning experience. It is becoming pretty clear that there is no “one size fits all” design and there can be advantages and pitfalls of each. However, I did want to do this deep dive on singleton, because I had seen it garner almost nothing but flak.

From the blog CS@Worcester – Alan Birdgulch's Blog by cjsteinbrecher and used with permission of the author. All other rights reserved by the author.