Author Archives: nealw5

Intro

Blogging for Software Capstone

From the blog CS@Worcester – Site Title by nealw5 and used with permission of the author. All other rights reserved by the author.

Hadoop

Installing Hadoop ran into issues initially with the fact that the document for installing is outdated. The mirror link for getting Hadoop is 404ed. The Java version moved onto 8 instead of using 7. I chose the latest stable version (2.7.3) on the Hadoop website to install.

Instead of using my account, I created a Hadoop account and sshed using that. Ubuntu does not Sudo by default so I had Sudo my Hadoop account or else anything accessing root will be blocked. I also ran into issues where NameNode and DataNode would not start up at all no matter what I did. After experimenting I realized that forcing a creation of hdfs folder with namenode and datanode was conflicting with what Hadoop wanted to do. Hadoop now automatically create both directories which allows my Hadoop to run smoothly.

The experiment I did with Hadoop is the word count example which WordCount.java will compute the word frequency of all text files in the HDFS input directory. Using the program, I began to understood the advantage of using Hadoop. The MapReduce job splits the data and processes them in multiple parallel processing. This seems especially advantageous working with multiple larger files.

http://hadoop.apache.org/

http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/

From the blog CS@Worcester – Site Title by nealw5 and used with permission of the author. All other rights reserved by the author.

Object Oriented Testing

In object oriented unit testing, select as units is the traditional way to unit test. In methods as units, we use mock objects with stubs and drivers that test the program and supply test cases to record results. We look at each individual method and then we see the result of the execution. When treating class as a unit, it solves the intraclass integration problem, but it introduces other problems such as ignoring inheritance. It is best used when little inheritance occurs and classes have internal control complexity.

When testing class as unit, if we flatten the class, original hierarchy must be restored. Also testing methods if added must be removed. If we are testing per method, we have to integrate the operations into the full class, and also integrate the class the method in with other classes.

http://www.ambysoft.com/essays/floot.html

https://www.tutorialspoint.com/object_oriented_analysis_design/ooad_testing_quality_assurance.htm

From the blog CS@Worcester – Site Title by nealw5 and used with permission of the author. All other rights reserved by the author.

Thread testing

Thread testing is an early system integration testing phase to verify key functionalities. Atomic System Function, ASF, is a system to show input and output event. Since ASF have port event as output and input, it shows a sequence of input and output event. This is used to represent the seams between integration and system testing. ASF graphs represent the system by showing which nodes are ASF and edges represent sequential flow. Source ASF shows as a source node while Sink ASF shows as a sink node. System thread links a source ASF to a sink ASF.

With thread testing, we can unify the three level if testing. Unit testing tests individual functions. Integration testing examines interactions among units, and system testing examines the interactions between the ASF.

http://www.softwaretestingstuff.com/2007/10/thread-based-integration-testing.html

http://www.guru99.com/thread-testing.html

From the blog CS@Worcester – Site Title by nealw5 and used with permission of the author. All other rights reserved by the author.

Mock Testing

Mock object testing is an interesting way to test special cases that mimic real objects for testing. The three phases of testing are expectation, exercise, and verification. Mock object testing is just one of the many testing doubles. There is dummy, in which objects are passed around but never used. There are Fake, where objects have working implementations, but have shortcuts for productions. Stubs provides canned answers to calls made during the test. Then there is Mocks where it is objects preprogrammed with expectations on what input to receive. Mock uses behavior verification in where we check to see if the order made the correct calls. Unlike other testing methods in which they use state verification, in where we determine the method by examining the state after the method was executed.

http://martinfowler.com/articles/mocksArentStubs.html

https://msdn.microsoft.com/en-us/library/ff650441.aspx

https://jtreminio.com/2013/03/unit-testing-tutorial-part-4-mock-objects-stub-methods-dependency-injection/

From the blog CS@Worcester – Site Title by nealw5 and used with permission of the author. All other rights reserved by the author.

Mutation testing

In the blog on mutation testing, the author talked about the need to basically test your tests. This mutation testing will deliberately introduce bugs into the code and verify that your unit test will fail. I definitely have written so bad unit tests before where it will execute my program and pass the test regardless of what the code actually does. This white box testing method is a very interesting way to ensure a higher level of reliability in your code.

On the other side of the spectrum, the Microsoft blog criticizes many projects that require an extreme high test coverage where it leads to a massive burden to development while catching few bugs. How many projects in real life have the time to not only unit tests completely but also do something like mutation testing. Let’s not even get started on working and testing on legacy codes.

Mutation testing: testing your test cases

https://medium.com/@s4n1ty/experimenting-with-mutation-testing-and-kotlin-b515d77e85b5#.er96cssn5

Stop writing bad tests

https://blogs.msdn.microsoft.com/ericgu/2016/10/09/stop-writing-bad-tests-write-only-the-tests-that-you-can-do-great/

Mobile development headaches

Android is great, I think it is a great mobile OS. However, the grievance of developing for it, I have my share of pain. The author complained about Gradle, the build system Google adopted for android, and it can be such a huge annoyance. I have heard people’s project waiting for 4-5 minutes for Gradle to compile. Every time you change something, it does a full Gradle build. However, this was still an upgrade over what existed before.

I have already written about my annoyance of outdated documentations and poorly documented API’s. Another issue that pops up is during emulator testing. Incompatibility with outdated Google Play Service with the emulator makes it so testing older API versions a pain. It makes testing apps that need to use the Google Play Service much more difficult.

The author says he’s quitting Android Development, it is not all bad. However, the cons of the Android that IOS seems to have no trouble with, it also has its many pros. The author seems to be more frustrated at Google’s nonchalance at the issues developer face with Android. Apple, being the closed system it is, have its own set of issues. Maybe we should all just switch to Windows Mobile as a protest. (That’ll be the day)

One developers good bye to Android:

https://raptureinvenice.com/its-been-real-android-why-im-retiring-from-android/

The Pain of IOS development

I have done some work on IOS apps, and the frustration of apple’s closed system is sometimes baffling. Testing for older IOS version is an extreme pain with Xcode. In order to test for older versions of IOS apps, you either need an older version of Xcode that only works on older version of Mac OS, or you need to have a device with older version of the IOS available. If you don’t have neither of those options, tough luck, either restrict the IOS version your app can run on or pray that no serious bug occurs on the older version of IOS.

Now the certificates and profiles is certainly the most frustrating thing about the app. Getting an application to run on a physical device may be one of the most confusing part of IOS development. In order to test on physical devices, you need to publish to that device. In order to do that, you need a developer profile with a specific certificate. That sounds easy right? Wrong, half of the time it’ll throw up an error on your profile, another half of the time your certificate is wrong. Apple must’ve focused all their time on writing the API, which Android really should read and study, and gave one intern the job to write the profile and certificate documentation. Half of the time you are wasting entire days browsing through Stack trying to fix the issue, which most of the time results in people telling you to do a clean reinstall.

People have always been clamoring for IOS to become more open sourced. Apple certainty have started to lean in that direction, with free developer accounts and the opening up of various IOS functions such as Siri. Unfortunately, Apple probably will never ever budge on certificates and profiles, it is their draconian way to enforce control on the apps that is allowed on IOS. Unless some major upheaval happens to Apple in the near future, we can only pray they streamline the profile and certificates.

Troubles for developing for IOS

https://medium.com/@Pier/why-i-hate-ios-as-a-developer-459c182e8a72#.o1suku996

StackOverFlow and Keeping Up with Changes

On the “Decline of StackOverFlow”, John Slegers laments the decline of contribution and the extremely draconian moderation that is pervading the site. He also criticizes the toxicity of its community towards newer members and their questions in general. It’s a concerning trend since Stack is when my code runs into issue or I have trouble debugging. One quick search on Stack and my problem will either be quickly solved or the answer will lead me to the right direction.

Even in the most popular topics on Stack, there is a huge issue of contents becoming outdated. There are numerous answers for language and framework that only applies to a version from a couple of years ago. Answers to deprecated methods is common especially in mobile development with Android in a constant flux of change and swift 3.0 officially being released just this month. Problem is even more frustrating in JavaScript, and many times it’s not even the fault of Stack. New versions are constantly being released breaking things left and right. In one blog I read last week for Angular2, it states that “Stackoverflow posts that are older than 6 weeks old are no longer relevant and probably deal with an issue from an old version that I’m no longer using.” The problem is exacerbated even further in much more obscure languages and also proprietary software. Where even getting a decent answer can be a matter of luck.

However, despite all of its flaws and people commenting on its “decline”, Stack is still a great website to solve your problem. Most of the common problems have been answered and is only one google search away. The strict moderation has led to mostly very well phrased questions and answers that not only solve the problem but help people understand the issue. Outdated answers (which are sometimes edited to reflect current version: thank you to all the people who goes back and update), even if outdated, can lead you in the right direction to understand your problem. Stack despite all the issues above, is still pretty damn good, and is and will continue to be an invaluable resource for all programmers.

The decline of stackoverflow:

https://hackernoon.com/the-decline-of-stack-overflow-7cb69faa575d#.2xjk5bv0z

A counter argument to stackoverflow decline:

http://techblog.bozho.net/i-stopped-contributing-to-stackoverflow-but-its-not-declining/

Understanding your code

In the blog written by Alex Naraghi on What I Didn’t Understand as a Junior Programmer, he states, “There is no excuse for not understanding the lines you have written, or you are debugging. There is no shortcut, even if it feels like there is.” It is much better to spend time mastering the entire system and understanding a project’s architecture. Spend the time to use good programming principles and design pattern. However, it’s interesting that what was warned in the article was what I did during the past summer.

I was tasked to develop prototypes for some basic mobile applications for the company. They wanted an application that can search their publications, and retrieve the info. I had some experience in Java and also Android Development, but let’s be frank, after not touch Android for a year, I only had a cursory understanding of what was going on. IDE changed, APIs deprecated, and with that a whole slew of new issue will pop up once coding starts. I was also tasked with learning Swift for IOS development, in which I had no experience in. They gave me some code some other intern wrote in the past, (which was written in Swift 1). Being an intern with not much to lose, of course I confidently said, “Yeah, I can do all of this.”

In the end, I pulled it off. 2 weeks were spent writing an Android application that can search, filter by categories, and use the microphone for speech to text search. The last one my boss was most impressed with, although it was basically a very few lines I read browsing stackoverflow. Swift was the same, 2 weeks of watching Youtube videos and browsing stack, I jury rigged an updated IOS app that allowed searching. My boss praised me, commenting that I gave an intern a month and he made me two apps. It didn’t matter the two apps were designed pretty terribly and running on janky code, but as my boss says as I cautiously brought this up, “hey it works.”

“Now, I believe that even if you fix a bug through fiddling around, it is more than likely you haven’t solved it.” The IOS app, I can only somewhat understand what I did and there were probably numerous issues with it. For that project, it was only a quick and dirty prototype. However, in the future, if I’m working on a huge code base, there is no way I can just fiddle my way around while not understanding the system without serious repercussion on future maintenance. The statement I hear often is, “as long as it works, nothing else matters”. A belief that I often read and now seen permeating through middle and upper level management. Alex Naraghi’s post just reaffirms my belief that in the world of programming, that the statement couldn’t further be from the truth.

Understanding the code and fixing problems as a programmer:

http://blog.alexnaraghi.com/what-i-didnt-understand-as-a-junior-programmer

Further interesting articles I read this week:

Al-Jazeera stealing codes from a web development company:

https://www.scrollytelling.io/al-jazeera.html

Learning Angular2 is excruciating (and JavaScript in general):

https://hackernoon.com/why-learning-angular-2-was-excruciating-d50dc28acc8a#.nb7772a3b

Chrome setting marking all HTTP to be unsecured in the future:

https://www.troyhunt.com/heres-how-broken-todays-web-will-feel-in-chromes-secure-by-default-future/

Knowledge Debt:

http://amir.rachum.com/blog/2016/09/15/knowledge-debt/