Author Archives: David MacDonald

Sprint 2 Retrospective

Summary

In this sprint, I mainly did the following things:

What Worked Well

This sprint, I worked a lot with Marcos on the backend. Overall, I think that we worked well together. This was the first time I spent most of my time working with someone else and it went well. We didn’t really run into any issues because we made sure to be careful with git. I now see why its recommended to add new features on a new branch. We were able to split up the work and stay in communication.

I also had a decent understanding of OpenAPI and Node during the work. It felt like I knew what I was doing in a sense.

What Didn’t Work Well

I think the team overall just had a lot to do outside of this class. I was able to get done what I needed to during class but if I had to work on things outside of class as well, I wouldn’t have had the time and the motivation at the same time. I think that overall as a team we didn’t get as much done as I expected because people were busy with other classes.

We also still sometimes feel lost and don’t really know what questions to ask. This applies mostly to the details of the design or implementation. However, it was a lot better than Sprint 1.

What Changes Could be Made to Improve as a Team

We need to get used to typing out more info for the cards. I’ve also noticed people aren’t always moving the cards they’re working on into the “doing task” column. Besides that, I think as a team we’re doing pretty well. We work together when its necessary or convenient and manage our own work otherwise.

What Changes Could be Made to Improve as an Individual

I could dedicate more time to this class overall and I could ask teammates if they need help rather than always waiting for them to ask for it.

From the blog CS@Worcester – The Introspective Thinker by David MacDonald and used with permission of the author. All other rights reserved by the author.

Apprenticeship Pattern – Unleash Your Enthusiasm

The key focus of this design pattern is the situation in which you have more enthusiasm for software development then the rest of your colleagues. Due to this, you end up holding back to fit with the group. I have a few experiences related to both sides of this. While I’ve never partaken in a software job, I have taken part in software courses. There have been plenty of times when I’m in a group project and the topic is so interesting to me I can just get to town and code almost everything. I guess rather than having it hold me back, I end up usually embracing it.

Then there are situations similar to that which I’m in now. It isn’t precisely a mirror for the apprenticeship pattern, but it’s pretty close. I find that when I have other things I need to do are when I’m most passionate about other things. For example, I’ll most intensely want to theorize about physics when I have an assignment due; however, once I actually have free time, I’m mostly content just wasting my time playing video games. Over the last month, I’ve become fixated with Sea. I had been working on creating a Minecraft Server manager in Node.js for my friends and I, then I moved onto creating a way of backing up my playlists. Then I realized it’d probably be easiest to do it in Python, so I started rewriting the code. In that process, I fell in love with Python.

I had used it before and enjoyed it, but I was almost something of an elitist. It had no strict types, it was easy to write. I treated it as if it were a beginners language. So I never really took the time to learn it. I rewrote my code in Python and learned a lot of the joys of Python such as context managers, list/set/dict/string comprehension, etc. (I just need to figure out the SQL commands and maybe I’ll eventually finish it). I had always been aware of the fact that Python running in a single threaded interpreter will never be able to perform as well as something like C. C by itself has a lot of joy to write in. Every language has its own personality you get to learn. That said, C can be tedious to write in and to read. That began my quest to create Sea – a version of the C language with Python-like syntax. I have become passionate about designing what the ideal language would be. Something modular, with high and low level features, and is easy to write and debug. Sea is just the first step in that.

I have found that I can so easily spend six hours straight coding, debugging, and refactoring Sea code. I can then go to bed and while I’m trying to fall asleep or even while I’m dreaming, I’ll be making design decisions. Classes that are otherwise fine can seem boring by comparison. Being able to just create something functional that has a clear use case feels great. At least sometimes, it can be really easy to share that enthusiasm with other students and it overall helps all of us.

From the blog CS@Worcester – The Introspective Thinker by David MacDonald and used with permission of the author. All other rights reserved by the author.

Creating a Transpiler

A transpiler is a program that converts code from one programming language to another programming language. This is comparable to a compiler, which is a transpiler that converts into machine code. It is also related to an interpreter, which behaves similarly, except rather than writing new code, it performs the code.

In my work on the Sea programming language I’m making, I took a long time writing a custom system for transpiling. However, while it succeeds at managing indentation pretty well, it makes actually transpiling statements much more challenging. So, recently I’ve gone back to the drawing board and have decided to pursue the classic model. If it ain’t broke, don’t fix it.

I’m working off of David Callanan’s Interpreter Tutorial. While it’s a very useful tutorial, the code is admittedly pretty poor, as it contains a few files with hundreds of lines. I’m also using Python exceptions to carry errors, since as far as I’m aware, Python has one of the safest exception systems (unlike C++). I can safely catch and handle exceptions to create useful messages for the user. The tutorial, on the other hand, is manually passing around errors from function to function. That said, the explanations are decent and it is a very useful tutorial. I’ll just have to make a lot of modifications and refactoring after each episode in the tutorial. That said, let’s go over how a transpiler works fundamentally:

The Process

The first step in transpilation is reading the source file. The lexer goes character by character and matches them to a set of predefined tokens. These tokens define a significant part of the syntax of a language. If it doesn’t recognize symbols, it can give an error that alerts the programmer. If there aren’t any errors, the lexer will go through the entire file (or files) and create a list of these matched tokens. The order of the list encodes the order that elements appeared in the file. Empty space and otherwise meaningless syntax symbols are not passed on.

Next, the list of tokens is sent to the parser. The parser will then go through the list of tokens and create an Abstract Syntax Tree (AST). This is a tree of tokens whose structure encodes the order of operations of the language’s syntax. In this stage, the order of the list is lost; however, that order isn’t important. What matters is in what order tokens should be. For instance, the list of tokens for 5+22*3 might look something like [INT:5, PLUS, INT:22, MUL, INT:3] and the list of tokens for (5+22)*3 might look like [LPAREN, INT:5, PLUS, INT:22, RPAREN, MUL, INT:3]. The ASTs for these token lists will look something like this respectively:

Created on https://app.diagrams.net/

Lastly, you then traverse the tree using depth-first-search (DFS), or more specifically, Preorder Traversal of the tree. This means we start at the root node and we the work our way down the left side and then down the right side. This is incredibly simple to implement using recursion. Each new node you check can be treated as the root to a new tree where you can then proceed to repeat the search. This occurs until the entire tree is traversed.

In this final stage, this is also where transpilers, compilers, and interpreters differ. Until now, the same code could be used for all three. At this point, if you want a transpiler, you use the AST to write new code. If you want a compiler, you use the AST to write machine code. If you want an interpreter, you use the AST to run the code. Notice this is why there is such a performance benefit to using a compiler over an interpreter. Every time you interpret code, assuming there is no caching system in place, the interpreter has to recreate the entire token list and AST. Once you compile code, it is ready to be run again and again. The problem then comes from compiled code potentially being more complicated for higher-level language features, and thus making it a pain to write a new compiler for every CPU architecture, due to different architectures using different machine instructions.

From the blog CS@Worcester – The Introspective Thinker by David MacDonald and used with permission of the author. All other rights reserved by the author.

From C to Shining Sea

The Snake

As of recently, I’ve been spending most of my personal coding time in Python. I enjoy a lot of languages and Python certainly isn’t for everything, but when you can use Python, boy is it a joy. As someone who strictly indents in any language, I love the indentation style of denoting blocks. Curly braces have their use, but the vast majority of the time, they’re purely redundant. The same goes for semicolons. I completely agree with the movement of programming languages towards spoken language. The main downfall of Python, comes from how high-level of a language it is.

Being a high-level language allows for it to be as convenient to write in as it is, however you are completely unable to use low level features. It also means Python’s performance is often much lower than that of C++ or other languages. Of course, everyone says that each language has its own use and Python isn’t meant for performance-intensive programs. But why not? Wouldn’t it be nice if there were a single modular language that had Python-like simple syntax with the features of JS, Python, C++, etc.

The Sea

Before I take on the task of creating such a language, I want to start smaller. Introducing Sea- It’s C, just written differently. I am currently working on a language called Sea which is effectively C, but with Python-like syntax. I say Python-like because much of the syntax of Python relies on internal data types. My goal is to keep Sea true to C. That is, no increase performance penalty; all of the penalty should be paid at compile time. That’s phase one. Start off with a more concise face for C. Then, I want to create libraries for Sea that take it one step further – introducing data types and functions innate to Python like range, enumerate, tuples, etc. Lastly, I want to use the knowledge I’ve gained to create the language to end all languages as described above.

I’m starting off with a Sea-to-C Transpiler, which is available on Github. In its present state, I am able to transpile a few block declarations and statements. I’m currently working on a data structure for representing and parsing statements. Once that’s made, I can add them one by one. The final result should look something like this:

include <stdio.h>
include "my_header.hea"

define ten as 10
define twelve as 12

void func():
    pass

int main():
    if ten is defined and twelve is defined as 12:
        undefine twelve
        // Why not

    c block:
        // Idk how to do this in Sea so I'll just use C
        printf("Interesting");

    do:
        char *language = "Python"

        print(f"This is an f-string like in {language}")

        for letter in language:
            pass

        break if size(language) == 1
    while true and ten == 11

    return 0

Once the transpiler is done, I want to create an actual compiler. I’ll also want to make a C-to-Sea transpiler eventually as well. I’ll also want to create syntax highlighting for VS Code, a linter, etc. It has come a surprisingly long way in such a short while, and I’ve learned so much Python because of it. I’m also learning a good amount about C. I’m hoping once I create this, there will never be any reason to use C over Sea. There are reasons why certain languages aren’t used in certain scenarios. However, I see no reason why certain syntaxes are limited in the same way. Making indentation a part of the language forces developers to write more readable code while removing characters to type. Languages should be made more simple, without compromising on functionality. That is my goal.

From the blog CS@Worcester – The Introspective Thinker by David MacDonald and used with permission of the author. All other rights reserved by the author.

Sprint 1 Retrospective

Summary

In this sprint, I mainly did the following things:

  • Learn Scrum – We all had to review how Scrum works and the different roles.
  • Understand GuestInfoBackend – I read through the Architecture to understand how the GuestInfoBackend is supposed to work.
  • Review Docker – I did research and found useful links to help us remember how to use Docker.
  • Docker Example – I created a basic docker example that we can go back to as a reference.
  • Learn Express – I did research into Node, Express, and OpenAPI for the backend.
  • Express Example – I created a basic express example with a few routes and such.

What Worked Well

I think overall, all of us in our team are all on the same page about this class. We understand what we have to do and no one is falling too far behind or going too far ahead. We’re good at helping each other when we need it and we try to put the team first.

Learning how cards work on the boards went really well I think. After that sprint, I think we all have a decent understanding of how to add labels, assign people, etc; moreover, we understand how to utilize the cards to aid workflow.

I also think we overall did a decent job of researching the materials we needed to and we have a decent understanding of the tools involved such as Docker, Vue, OpenAPI, etc.

What Didn’t Work Well

The biggest difficulty comes from the fact that this is a class rather than an actual job. That means we only have 3 hours of class time per week, and then we need to allocate personal time to work. I find this especially challenging when I have other homework to do, and other personal projects I want to work on such as creating a version of the C programming language that has python-like syntax. The biggest limiting factor to all of this is time, with the second one being motivation/interest. I find it incredibly easy to spend an entire day working on something I really enjoy and I find it incredibly hard to work for even half an hour on something I don’t care about. This falls somewhere in between the extremes.

Another thing that adds difficulty is, at times, it feels like we have no idea what we have to get done. The problem is that when that happens, we don’t even know where to start asking questions to figure out where to go.

What Changes Could be Made to Improve as a Team

Overall, we could probably work together more both in class and outside of class. Most of our work has been done mostly alone unless necessary, however there are some times when it would be convenient or useful to work with more people at once.

We all could do better at managing time, as previously mentioned. Using card due dates, we can help keep each other working in a decent time frame.

What Changes Could be Made to Improve as an Individual

I could be more vocal in classes, as well as work more outside of class. This applies to both working more on this class in my own time, as well as working on my other classes in my own time.

From the blog CS@Worcester – The Introspective Thinker by David MacDonald and used with permission of the author. All other rights reserved by the author.

Apprenticeship Patterns – Revision

Context

The following post will be in reference to Apprenticeship Patterns: Guidance for the Aspiring Software Craftsman by Dave Hoover and Adewale Oshineye chapter 1 and the introductions to chapters 2 to 6.

This revision is to decrease word count.

Opinion

“Most people won’t have an opportunity to work in a formal apprenticeship where they are being mentored by software craftsmen. In reality, most people have to claw and scratch their apprenticeships out of less-than-ideal situations. They might be facing overbearing and/or incompetent managers, de-motivated coworkers, impossible deadlines, and work environments that treat novice developers like workhorses, storing them in small, rectangular stalls with a PC and a crippled Internet connection. (Apprenticeship Patterns)”

I have many options for what to discuss, so I’ll be discussing the current state of the job environment. Due to numerous factors, students are graduating college and finding that, in order to get an entry level job, they need experience. In order to get experience, they need an entry level job. It’s a catch-22.

Every issue is infinitely complex. There are numerous variables that have caused this situation to exist and it would be impossible to adequately address all of them. So, I will focus on a few such as the abundance of college degrees. Just as every issue is complex, every good thing has bad aspects and every bad thing has good aspects. Most people can agree that a population with college educations is a positive. Education provides opportunities. However, this is bound to also have unforeseen consequences.

Employers want to hire people that will be the best for the position. This is essential for the survival of the business and the economy at large. Not everyone is fit to be mathematician, for example. So, employers must discriminate between candidates in order to determine those who are best fit for the position. One large determining factor is a college degree. Having a degree in a subject implies familiarity and some level of expertise in said subject. However, it doesn’t end at employers. Those applying for positions realize the advantage of a degree and so an entire generation of parents have been pushing their kids to get a degree, with the goal being a decent job.

This drive for degrees creates a huge supply of degrees and with the demand remaining roughly the same, the value of a degree decreases. If everyone has a degree, an employer can’t use it as a discriminating factor. So, they move onto others such as prior experience. The problem arises from the lack of supply of jobs and the huge competition for those jobs. We find that thousands of students are going to college now to get a degree they might not even use and end up in massive debt.

This is by no means a simple issue. There are a handful of helpful practices that can improve the situation. Pushing trade schools as well as colleges can lead kids down a successful path and decrease the supply of degrees, making them worth more. Government can decrease regulation to allow more small businesses to compete for workers. However, there will be no magical single solution to this problem and what it really requires is a societal discussion about college degrees.

Work Cited

“Apprenticeship Patterns.” Accessed February 22, 2021. https://learning.oreilly.com/library/view/apprenticeship-patterns/9780596806842/#toc-start.

From the blog CS@Worcester – The Introspective Thinker by David MacDonald and used with permission of the author. All other rights reserved by the author.

Apprenticeship Patterns

Context

The following post will be in reference to Apprenticeship Patterns: Guidance for the Aspiring Software Craftsman by Dave Hoover and Adewale Oshineye chapter 1 and the introductions to chapters 2 to 6.

Opinion

“Most people won’t have an opportunity to work in a formal apprenticeship where they are being mentored by software craftsmen. In reality, most people have to claw and scratch their apprenticeships out of less-than-ideal situations. They might be facing overbearing and/or incompetent managers, de-motivated coworkers, impossible deadlines, and work environments that treat novice developers like workhorses, storing them in small, rectangular stalls with a PC and a crippled Internet connection. (Apprenticeship Patterns)”

This is an incredibly true statement. Due to the current state of the economy, it is very hard for people to gain work experience in any field, let alone in software development. One will frequently see requirements for years of experience in an entry level position, but how can one achieve years of experience if every entry level job requires it? In my opinion, this is the product of many things; moreover, it is an incredibly complex issue, as all issues are. One such thing is the abundance of college degrees. As with many of these possible causes, they are often both positive and negative. Every good action has negative consequences and every negative action has positive consequences.

The positive that comes with the abundance of college degrees is likely self-evident – average people are educated and have more opportunities. However, consider the issue of student loan debt. Many people put themselves into massive debt to get a degree they might not even use. Anyway, it is basic economics at play; you cannot fight supply and demand. The hiring process is simple. Now, the world we live in isn’t perfect but suppose it were an actual meritocracy where everyone has equal opportunities. How can we hire? If we assume that someone who has spent years in college dedicated to studying a topic we work with, we can discriminate based on degree in order to discriminate on skill set. It is important to note that discrimination is not inherently bad. In a meritocracy, discrimination is necessary; in fact, in any value hierarchy where object A has more value than object B, discrimination is necessary. The problem with discrimination is when we discriminate based on irrelevant characteristics. For instance, race has no influence on a person’s ability to be a good doctor. So it is unfair to discriminate based on race when deciding a doctor. However, suppose you need brain surgery and you have the choice of a doctor with twenty years of experience, or a new hire without any experience. In that case, in order for society to function at all, it must be considered fair to discriminate based on experience since that directly affects the chance of success and, in this case, survival. Thus, jobs fairly discriminate based on perceived ability and understanding to find the best for the job. However, the more people that have a degree, the less worth a degree has relative to employment. Having a degree is less of a discriminating factor and so jobs have to create other means of determining skill – hence the absurd amount of experience required.

It is a catch-22; employers use college degrees to discriminate to determine those best fit for their position so average people push their kids to go to college. Then, college degrees become worth less but people still think they are necessary so despite the immense cost, they push their kids to go anyway. Obviously, college is not bad. However, their is a failure to have the necessary discussion with kids. They have to decide whether the career they want is going to be worth the investment. They also need to consider that they might not even find a job in their field after college. It would help the issue if parents and schools viewed college as more of a tool rather than a necessity, and if they introduced kids to trades. Trades also aren’t for everyone, but they are a very founded career path. What society needs overall is a balance and a discussion. In my estimation, the entire education system needs an overhaul; but that is for another blog post.

I would go into the other things I think are causes of such a job environment, but I’ll save that for another day. Instead, I’ll focus on the immense power of environment. First and foremost, people like to view their environment as separate from their minds. However, just think about it. As an example, when you’re lazy, you create clutter in your environment and when your environment is cluttered, it can cause stress or discomfort. Your environment affects your mood and your mood affects your environment, so the distinction between the two isn’t as clear as people might like to think it is. A good working environment is crucial to doing good work. Think back to primary or secondary school and think about the old books and uncomfortable desks with pencil carvings in them. It might go without saying but, if you want to improve your work, put some effort into ensuring your work environment is designed for you to be productive.

Work Cited

“Apprenticeship Patterns.” Accessed February 22, 2021. https://learning.oreilly.com/library/view/apprenticeship-patterns/9780596806842/#toc-start.

From the blog CS@Worcester – The Introspective Thinker by David MacDonald and used with permission of the author. All other rights reserved by the author.

Introductory Post for CS-443 and CS-448

Hello everyone! This is just a quick post to say I’ll be writing periodic blog posts for both CS-443 and CS-448 this semester. I’ll be looking forward to writing for all of you. Feel free to also browse any other topics I’ve discussed and past classes I’ve written for as well!

From the blog CS@Worcester – The Introspective Thinker by David MacDonald and used with permission of the author. All other rights reserved by the author.

It’s Easy to Break Promises

Context

I’m currently writing a Node.js package for a wrapper I’m calling a curfew-promise. It’s simpler than I thought, but it is still worth doing in my opinion. The package exports a single function that returns a promise. The promise has “3” parameters with the function header being: (curfew, func, ...args). The promise then performs func asynchronously (so func itself can be sync or async). It passes ...args to that function when it’s called. Lastly, the key idea here is that if func takes longer than curfew milliseconds to complete, the wrapper will become a rejected promise.

At first, it seemed like a tough task, until I discovered the built in Promise.race() function which creates a promise which we can refer to as the racePromise. That function then also takes in multiple other promises. Whichever passed in promise resolves or rejects first, its value is then passed onto racePromise. I can achieve my curfewPromise then by creating one promise that runs func and one that rejects after curfew has passed. Whichever finishes first becomes the value of the promise the function returned. The only way I’ve found to pause an async operation in JavaScript is via await new Promise((resolve) => { setTimeout(resolve, duration); }); This line, when placed into any async function, will pause operation until duration has passed. You can also remove the redundant brackets and shrink it down to await new Promise(r => setTimeout(r, duration)); , but I prefer to be more consistent.

JavaScript is NOT Multi-Threaded

JavaScript runs in a single thread. The way it pretends to be multi-threaded is by switching back and fourth between tasks to give all of them a little bit of time until they all finish. JavaScript is also an interpreted language. This means while trying to test my package, I had a lot of trouble. I spent a few hours trying to understand what was going on and now that I have, I’m going to write about it to save you trouble.

The Code

I was completely convinced that JavaScript itself was just broken. Here is my package, at least in its current state: (If you’d like to use this or see the modern version, check the GitHub page (soon to also be on npmjs.com))

module.exports = (curfew, func, ...args) => {
    let waitId;

    async function wait() {
        await new Promise((resolve) => { waitId = setTimeout(resolve, curfew); });
        return Promise.reject(new Error("Curfew of " + curfew + "ms have elapsed."));
    }

    async function perform() {
        const value = await func(...args);
        if(waitId) clearTimeout(waitId);
        return Promise.resolve(value);
    }
    
    return Promise.race([wait(), perform()]);
}

wait() creates a promise (using async notation, since from what I’ve read that is more ideal than direct promise notation) which waits until curfew has passed, which then rejects with a useful stack trace message. You’ll notice I’m also storing waitId, which is the ID of the setTimeout call. This way, if func finishes before curfew, I can cancel the timeout and not waste performance. I’ll also be looking into ways to create cancellable promises. I’m aware there are already packages that do this, but I think I’ll benefit from doing it by hand. I could make wait() a synchronous function that simply returns a waiting promise that calls setTimeout for reject, but I chose making it like this because then it matches the form of perform() (two async functions), and it allows me to write two lines that are visibly neat rather than trying to force everything into one line.

perform() creates a promise that waits until func is finished. Once it is, it’ll attempt to stop the waiting promise. If the promise is still waiting, it’ll stop it. If the promise has already rejected, it will do nothing. Then, the function returns a resolved promise with the value from func. I chose to write return Promise.resolve(value) instead of return value, which I understand to be the same thing, for consistency once again, and I think it makes the code more readable overall.

Lastly, I am creating the promises off of these functions and making them race. The function returns a promise that resolves or rejects whenever the first of the two – wait() and perform() – finish.

The Problem

I was able to narrow down the problem to this line:

        const value = await func(...args);

All of my tests had worked until I did something like the following:

const curfewPromise = require("./index");

// Broken Promise 1
console.log(curfewPromise(10000, async () => { 
    for(i = 0; i < 1000000; i += .001);
}));

// Broken Promise 2
console.log(curfewPromise(10000, async () => { while(true); }));

I found that for broken promise 1, despite the fact that curfewPromise returns a promise, it wouldn’t finish until the for loop finished. Even if I set the curfew to 0, it would take until the for loop completed. Broken promise 2 would never even end. I purposefully wanted to test these edge cases, where something would actually take a long time for ever (a good example for why I want the ability to cancel a promise).

The problem that I realized so frustratingly last evening is, and say it with me, JavaScript is NOT multi-threaded. Yes, when you run an asynchronous function, you can do other things in the meantime while it finishes. But as far as I can tell, JavaScript does this on a line by line basis. If you have a single line that is very slow or infinite, JavaScript will start running it, as that’s how asynchronous functions work, and then after some progress, it will move on. The thing is, you can’t make progress until that line ends. In the case of an infinite while loop, it will never end. In the case of a very slow one-line for loop, it can’t move on until the for loop is finished. Promise.race() cannot return a promise, so curfewPromise cannot return a promise. Once the for loop actually ends, it’s a gamble to decide which non-neutral-state promise will be chosen to have won the race. (Also keep in mind when using very small curfews, there is internal delay and it may not behave how you expect).

Conclusion

JavaScript is not multi-threaded. Make sure you keep that in the back of your mind. I was inadvertently testing for a case that shouldn’t ever even happen. While it was frustrating trying to figure out what was wrong, I still think that this kind of struggle is necessary in both life and learning. I always learn the most in coding when I’m trying to find out how to do something and I have 20 tabs open.

While this is a simple package that does something many developers probably can do easily, I still want to spend time making it and even upload it to npm. I think even though it’s simple, the function still helps clean up syntax to improve readability and reduce lines. I also think making it an external package helps to reduce complexity in your projects and helps me use it in multiple projects. Maybe someone can also look at it for help when learning how promises work.

At the end of the day, promises and asynchronous functions are an incredibly valuable tool in JavaScript and the illusion of multiple threads helps in most situations. Just be careful to not break your promises.

From the blog CS@Worcester – The Introspective Thinker by David MacDonald and used with permission of the author. All other rights reserved by the author.

Running Bash Commands in Node.js

Disclaimer:

This is not explicitly a “guide”. There are far better resources available on this topic; This is simply a discussion about the child_process module.

Anecdotal Background

I’ve been in the process of creating programs to automate the running of my Minecraft Server for my friends and I. In that process, I’ve created a functional system using a node server, a C++ server, and a lot of bash files. While it works, I can’t be sure there aren’t bugs without proper testing. Trust me, nothing about this project has has “doing it properly” in mind.

When learning front-end JavaScript, I recall hearing that it can’t modify files for security reasons. That’s why I was using a C++ server to handle file interactions. Little did I realize node can easily interact with files and the system itself. Once I have the time, I’m going to recreate my set up in one node server. The final “total” set up will involve a heroku node server, a local node server, and a raspberry pi with both a node server (for wake on lan) as well as an nginx server as a proxy for security for the local node servers.

Log

As a bit of a prerequisite, I’ve been using a basic module for a simple improvement on top of console.log. I create a log/index.js file (which could simply be log.js, but I prefer having my app.js being the only JavaScript file in my parent directory. The problem with this approach, however, is that you end up with many index.js files which can be hard to edit at the same time).

Now, depending on what I need for my node project I might change up the actual function. Here’s one example:

module.exports = (label, message = "", middle = "") => {
    console.log(label + " -- " + new Date().toLocaleString());
    if(middle) console.log(middle);
    if(message) console.log(message);
    console.log();
}

Honestly, all my log function does is print out a message with the current date and time. I’m sure I could significantly fancier, but this has proved useful when debugging a program that takes minutes to complete. To use it, I do:

// This line is for both log/index.js and log.js
const log = require("./log"); 

log("Something");

Maybe that’ll be useful to someone. If not, it provides context for what follows…

Exec

I’ve created this as a basic test to see what it’s like to run a minecraft server from node. Similar to log, I created an exec/index.js. Firstly, I have:

const { execSync } = require("child_process");
const log = require("../log");

This uses the log I referenced before, as well as execSync from node’s built in child_process. This is a synchronous version of exec which, for my purposes, is ideal. Next, I created two basic functions:

module.exports.exec = (command) => {
    return execSync(command, { shell: "/bin/bash" }).toString().trim();
}

module.exports.execLog = (command) => {
    const output = this.exec(command);
    log("Exec", output, `$ ${command}`);
    return output;
}

I create a shorthand version of execSync which is very useful by itself. Then, I create a variant that also creates a log. From here, I found it tedious to enter multiple commands at a time and very hard to perform commands like cd, as every time execSync is ran, it begins in the original directory. So, you would have to do something along the lines of cd directory; command or cd directory && command. Both of which become incredibly large commands when you have to do a handful of things in a directory. So, I created scripts:

function scriptToCommand(script, pre = "") {
    let command = "";

    script.forEach((line) => {
        if(pre) command += pre;
        command += line + "\n";
    });

    return command.trimEnd();
}

I created them as arrays of strings. This way, I can create scripts that look like this:

[
    "cd minecraft",
    "java -jar server.jar"
]

This seemed like a good compromise to get scripts to look almost syntactically the same as an actual bash file, while still allowing me to handle each line as an individual line (which I wanted to use so that when I log each script, each line of the script begins with $ followed by the command). Then, I just have:

module.exports.execScript = (script) => {
    return this.exec(scriptToCommand(script));
}

module.exports.execScriptLog = (script) => {
    const output = this.execScript(script);
    log("Exec", output, scriptToCommand(script, "$ "));
    return output;
}

Key Note:

When using the module.exports.foo notation to add a function to a node module, you don’t need to create a separate variable to reference that function inside of the node module (without typing module.exports every time). You can use the this keyword to act as module.exports.

Conclusion

Overall, running bash, shell, or other terminals in node isn’t that hard of a task. One thing I’m discovering about node is that it feels like every time I want to do something, if I just spend some time to make a custom module, I can do it more efficiently. Even my basic log module can be made far more complex and save a lot of keystrokes. And that’s just a key idea in coding in general.

Oh, and for anyone wondering, I can create a minecraft folder and place in the sever.jar file. Then, all I have to do is:

const { execScriptLog } = require("./exec");

execScriptLog([
    "cd minecraft",
    "java -jar server.jar"
]);

And, of course, set up the server files themselves after they generate.

From the blog CS@Worcester – The Introspective Thinker by David MacDonald and used with permission of the author. All other rights reserved by the author.