Evidence of Activity on GitLab
- Meeting with Customer to Identify Categories
- A
.js
file namedcategories.js
was built to help my peers start testing different category structures.
- A
- Data Extraction for TeaSparty Project
- The USDA FoodData Central downloadable data was extracted to build a comprehensive database.
- Database Optimization
- The original database file was over 3.2GB and contained multiple unnecessary fields.
- A Java file was developed to remove unnecessary fields such as
foodNutrients
,nutrients
,rank
,protein
, and their respective IDs. - Successfully reduced the database from 3.2GB to 0.7GB, making it more manageable for our project.
Reflection on What Worked Well
- Customer Collaboration
- Meeting with the customer to identify category requirements ensured that the project met their needs effectively.
- The creation of
categories.js
provided a starting point for peer testing, helping streamline future work.
- Database Optimization
- By removing unnecessary fields, the database was significantly reduced in size, improving efficiency.
- The pre-cleaning process helped identify redundant fields early on, making subsequent steps more manageable.
Reflection on What Didn’t Work Well
- API Call Extraction Issue
- Initially, I attempted to extract the database via API calls but encountered a limitation: API calls only supported retrieving 100 items at a time, whereas there were over 100 pages to extract.
- Researching solutions, I found that automation was the best approach, but due to my lack of expertise in automation, I couldn’t implement it effectively.
- Challenges in Categorizing Products
- A Java file was created to categorize all products, but it was crashing due to issues with object field descriptions containing multiple nested quotes.
- Example issue:
"description":"ORIGINAL SWEET & SMOKY BAR \"\"B\"\" \"\"Q\"\" SAUCE, ORIGINAL"
. - As a workaround, I manually cleaned some of the problematic entries but am still looking for a better solution.
Reflection on What Changes Could Be Made to Improve as a Team
- Better Communication
- Ensuring consistent updates within the team would help synchronize our work and avoid redundant efforts.
- More frequent check-ins to discuss roadblocks and potential solutions could improve efficiency.
Reflection on What Changes Could Be Made to Improve as an Individual
- Increase Work Hours on the Project
- Spending more time on the project will allow my peers to see my progress and provide feedback earlier.
- Dependency on Others’ Progress
- I am currently waiting for Alex to finish the dev container. In the meantime, I am focusing on cleaning the
db.json
file so I can upload it as soon as possible.
- I am currently waiting for Alex to finish the dev container. In the meantime, I am focusing on cleaning the
Apprenticeship Pattern: “Expose Your Ignorance”
Summary of the Pattern
The “Expose Your Ignorance” pattern from the Apprenticeship Patterns book emphasizes acknowledging one’s knowledge gaps and actively seeking to learn. It encourages transparency about what you don’t know while making efforts to bridge those gaps.
Why This Pattern Was Selected
During this sprint, I faced challenges with API automation and data cleaning that slowed my progress. If I had openly acknowledged my lack of expertise in automation sooner, I could have sought help from peers or external resources earlier in the sprint. This pattern aligns with my experience because it highlights the importance of being upfront about knowledge gaps and taking proactive steps to fill them.
How This Pattern Would Have Changed My Behavior
Had I followed this pattern earlier in the sprint:
- I would have sought advice from team members or forums on API automation instead of spending excessive time trying to figure it out alone.
- I would have explored existing libraries or tools for handling JSON parsing issues more efficiently instead of resorting to manual cleaning.
Conclusion
This sprint provided valuable lessons in data handling, API limitations, and the importance of seeking help when needed. Moving forward, I plan to improve communication within the team, allocate more time to the project, and be more proactive in addressing technical challenges. Applying the “Expose Your Ignorance” pattern will help me become a more effective contributor in future sprints.
From the blog Discoveries in CS world by mgl1990 and used with permission of the author. All other rights reserved by the author.