A Journey into Open-Source

RSS Feed
A Journey into Open-SourceSource: https://www.pexels.com/photo/paper-in-vintage-typewriter-12220470/

In March 2023, I took a significant step by open-sourcing some Node.js scripts developed for Contentful at my former company. This article explores the backstory, the process, and the insights gained from making proprietary code available to the public.

Starting the Journey

Open-sourcing was not an impulsive decision; it had always lingered in the back of my mind. It all started around six years ago when I first discovered Contentful, a Blog-as-a-Service platform. It's a very straightforward tool: you define a Blog structure (Content-type), and then use a series of APIs and SDKs to integrate into your project or website. What I have always appreciated about Contentful is their API design, reflecting the considerable effort behind it to maximize usability. This very blog is based on Contentful.
About three years ago, I joined my previous company, Atida, and was pleased to learn that they were using Contentful for the E-Commerce platform we were building. As a PHP Software Developer, this presented an opportunity to transition to new technologies (Node.js and Next.js) using a familiar platform, which was quite refreshing after having primarily worked in PHP.
And to be fair, one other reason I was happy to work with Contentful was that, for how much I appreciated (and still do) the platform and the design, a couple of things always bugged me:
  • The first one is that the 'Environments' in Contentful are a great thing for building and testing stuff, but it is (or at least it was) claimed that 'Environments' are just like Git 'Branches'. And that is actually not true. I get it, it's to make a user relate and understand those Environments better, but in reality you can't manage them really like Git Branches, not even close. But it's sort of possible (with some work).
  • The second thing that bugged me was kind of what makes me like Contentful. They give you a broad set of tools, SDKs, APIs, tutorials and so on, but the last '20%' is left to you, the developer, to personalize and figure it out. It's a fair trade, since being too specific would inevitably make it less useful for different contexts and would bring an additional layer of technicality. In addition, it would be difficult for them to shape and change their APIs over time to accommodate more specific use-cases. But nonetheless, although building that last-mile is exciting as a Developer, I can't shake the feeling that things could be better.

Building is the key

So, when I got the opportunity to work with Contentful in a more serious way, I was more than happy to do it, and in almost 3 years in my last job, I decided that I would 'fix' these things by myself. It's not like I had a clear plan from the beginning, but I got to start shaping it pretty soon. The first things I worked on, in the company, were simple stuff, and all based on the CMA (Management API) of Contentful. The main drive was to make the CI/CD deployment and Contentful work seamlessly together. To achieve that I worked on some scripts:
  • An 'environment' exporter, built on top of Contentful own exporter library, to do something very simple: export a Contentful Environment into a ZIP file, when running the script in the CI/CD. The script would know the right Contentful Environment, and export both the JSON with all the entries and a folder with all the images/assets. It would then ZIP everything and make it available as a CI/CD Artifact.
  • A 'migration' system that is Environment dependent, and doesn't need a centralized server to know which migration has run and which one has not. Similarly, the second step was to save the migration 'number' inside Contentful itself (you just need a key-store Content-type), like many databases do, and then just save JavaScript Content-type migrations inside a folder, using an incremental number. That was easy enough to implement, but what I didn't foresee was how useful it was to be able to have migrations applied in the CI/CD during deployment. It was a step further into really starting to treat the Contentful Environments as Git Branches.
  • The third script was a 'release management' tool. Meanwhile the first 2 scripts were kind of utilities that anybody could write, the third script was a more flashed out CLI script to help really (or I would say, finally) treat those Environments like Branches. Duplicate, verify them, sync their schedule (that is usually not duplicated between Environments), create aliases and whatnot. This script has been really what made it possible for us, at Atida, to work with Contentful in a way that was neither disruptive or invasive for content editors.
Me and my (amazing) colleagues have built, over the years, more scripts, but I won't dive into them into this article. But the journey of working with Contentful was extremely satisfying. But wait! You said at the beginning of the article that the journey started in March 2023? Let's clear this up. I started working with these Contentful scripts probably around May/June 2021, but in March 2023 I finally got the approval to start making these scripts Open-source. It wasn't easy to convince the management, since I understand the value this sort of stuff can have for a company. But luckily Atida, and in particular of my direct manager at the time,Shaya Pourmirza, have been very supportive of this, and instrumental into making it a reality.
If you are still reading. So how did it go? The first thing I did, I took the Node.js scripts I had and started looking at them with the perspective of a person that would eventually use it for the first time. And the scripts were... kind of unusable. Wait? What? Yes, you heard it right. The scripts I worked on for two years, had a lot of problems (at least for the imaginary standard I wanted to uphold):
  • They were not meant to be re-used. As any professional developer reading, you can surely relate. In my mind everything was perfect, but in reality the code had some trade-offs: hard-coded paths, constants that we only would use; it had methods that were not consistent and it wasn't following any particular 'standard' in the way some things looked (a method could have more logging, or the order of the parameters different from one another). And this is kind of natural. The first scripts and functions didn't have the quality of the latter ones. But when you work, you have to deliver, so you can't just try to perfect the code again and again.
  • The second problem was realizing that 'people will look at my code'. It seems silly to say, but coming from a PHP background I started reflecting on the fact that I'm not probably the most experienced Node.js developer, and so it kind of made me think twice before showing the code we made. Surely I didn't want people to think I didn't put the effort into making the code look good or be readable enough.
  • The third problem was that I never released an NPM package. So that was also a thing: it's easier to do something you know. So trying to think what to release, but being in the darkness about how to do it, surely didn't help with the general feeling of 'wanting to do some Open-Source'.

What did I learn?

  • Coding is fun - Having done professional Software Development for 15 years, I do love to code, but I would lie if I would say that every moment of every day at work is fun. It's not! Don't get me wrong. I feel blessed that I get to do a job that I like, but not all the job is equally fun, and not all the job is just coding. You have dailys, meetings, road-maps, refinements, and whatnot to 'distract' you from coding (some of these meetings help you, but still). And even when you have to actually do the job, sometimes you have no clue, you have to figure out what to do and you don't know the best way, cause you haven't done it yet. There is a 'procrastination' / 'digestion' process that makes you not really coding, just processing all the information. So some parts of the job are not fun, they are not meant to be. On the other end, when coding for yourself, on your own terms, it is a lot of fun. It's like when you were trying things out before you ever became a professional. It's refreshing and it sparks a lot of creativity.
  • But it's really hard - But it's still really hard. The reason(s) are that nobody really tells you what to do, but it also means that it is really easy to go down the pitfall of 'never finishing' something. Maybe you want to make it perfect, or just add a new method. So the hardest part is probably to remind yourself that even if it's free and it's fun, you can't just do that forever. You should treat it like a job. At one point, you need to decide that the code is ready, stop over-engineering and just release it. Also rewriting things like 10 times just to make them better, starts to wear you down and makes you lose clarity. Maybe that's how a 'scrum-style' sprint should look like. You deliver, you take your eyes off of it for a couple of days, and then you work on the next version. You'll probably see things more clearly.
  • Help from ChatGPT - I'm a Software Developer, but I use ChatGPT to help me code or figure out things. And I have no shame to say that is probably one of the best things as a developer that I could do. I started using ChatGPT to help me optimize and improve the code I was going to release, and it showed me some nice tricks or ways of doing things that I wasn't aware of before. So, for the people that are scared AI will take your job, it won't, but you should be using it for your own personal growth to make you become a better developer. And once you learn a new way of doing things, or just how to write a better JSDoc, you can then use that in all the rest of the code. That's what I did for my Contentful Lib Helpers.
  • Still TO-DO - But as I said at the beginning, at one point you have to release. And I feel it's better to release something useful and still have things TO-DO, than wait till everything is done. I know, I should add more tests, and probably a better way of handling the messaging, or just some optimization and improvements that will make the whole library better, but for now, this is a good trade-off. Maybe at one point I will rewrite everything using Typescript - we'll see.
For the technical things I learnt, I will do a separate article (or more than one), where I will share some of the code and the process it went through.

Special Thanks to...

I can't thank enough these companies or people that have helped me achieve this little step towards Open-Source:
  • Contentful - Well, Contentful just for creating the platform and all the amazing ecosystem.
  • Atida - My former company. An health/beauty products E-commerce, that supported this initiative.
  • Shaya Pourmirza - My former manager, amazing person and great professional.
  • Aldo Fernández - Probably one of the best FE developers I ever worked with. He thought me a lot and these scripts are heavily his contribution as well.
Published: Thursday, May 9, 2024, 11:08 AM - Updated: Sunday, May 12, 2024, 06:34 PM
Fabrizio C.
Fabrizio C.

Fabrizio C. is the founder of AsPlanned, and is a Software Developer with more than 15 years of experience in Software Development. He currently works for Zeeg - The Scheduling App.

Subscribe to Newsletter