I can't believe the blog engine that supports this web site has been implemented three times. It took almost five years. I like to say it is over. But the maintenance to make it better is never ending. I like to say, this is the best and proudest work I have ever done. I put a lot of effort (sweat and tear) into this blog engine. It is, as it stands, the proudest achievement of my life. This post documents how I have make it.
To be honest, I was lost for a long time. Between 2003 to 2013, I didn't do anything useful. I was lost and I didn't know what I wanted to accomplish with life. After the graduate school, I had some time so I decided to write the blog engine. The earliest version was started sometime in 2014. I could have choose some easier route, like join Facebook, or use WordPress. Both approaches seemed stupid. I joined Facebook for a month or so, and I didn't get it. I did join Twitter for a few days, and I didn't get it. I quit both. I was trying to get into Instagram. That "shit" couldn't run on a laptop. Blogging is the only thing that make sense. I tried WordPress for a while. And I didn't pick it up, because I didn't like PHP. Java is the language for me. I wanted to do something with Java. By 2015 or so, cloud based computing was finally available for common folks like me. It was so easy to sign up a small Linux server and have anything I made run on it. Now I know, my decision to do this is the right one.
When I started on this version, the third version, I had a revelation. It is called small data, private data. Small data, private data means that the data I put out online is my data, I have total control over them. The emphasis of small is that for an average blogger, 5 years worth of blog posts would be around 500 or so, it is extremely tiny in comparison to the massive personal data big corporations held. The emphasis of private is that one should have total ownership of the data. The thoughts and feelings, the opinions, and etc should be in total control of the owner. The only way to have both is by self-hosting the application and contents. By the time I was deep into the development of this 3rd version, it was a mission to me to write it to suit my own needs, and allow me to have total control of contents.
The very first version of this was developed with Java 1.6 and Spring 2. It was designed as deploy-able war archive. The development took four months. At the time, there were significant issues with the application. The admin user can add blog posts, but quite a bit of the page contents are hard coded in text files, then they are hard coded to display in the MVC controller. At the time, I know I have to rewrite it because the design is crap. I didn't want to fix it. I feel it is better to push it all down and rebuild from bottom. This is my style, when thing gets messy, I push it down and rebuild it. In the process, better practice are adapted and the next version would be better than the first.
Shortly after the first version went live, I started the rewrite. The initial idea is that I should be able to do everything with Spring MVC. I moved the code from Spring 2 to Spring 3. When I got into the core functionalities, I realized I was wrong that Spring MVC alone is enough. A lot of administrative operations has to be supported by JavaScript. That is the point when the development of second version got in the crooked path. I learned that a lot of UI interactions had to happen in browser. This means I have to use JavaScript extensively. I chose JQuery because it was what I knew at the time. Then after I went deep with the development and testing. I realized I didn't know enough about JQuery. By that time, I should have ended the development. push it over then rebuild from scratch, with something better. I didn't and continued until it was completed. Then I drifted off on something else for a while. Then I decided to push it over and redo everything again. The entire experience is not a total waste of time. I learned a lot about JQuery, which is enough for me to realize I need something better.
During the development, I found a lot of other issues, design issues and inadequacies. There is one where I built user log in functionality, and then realized there is no need for user to login. There is also the issue of HTML editing, where I found it is extremely hard to preview the changes I have made. All the coding with JQuery was huge mess, And all the source code are packaged in one project. All these decisions that were made, was not the smart ones. When it is finished, I was not going to spent time and maintain it or extend it. It was too messy, and I was too exhausted to fix it from top to bottom.
After version two went live, I realized a couple things:
If I don't work enough on the second version, I would never realize these subtle things. And the ways these changes can be done are all incorporated into the third version. When I started, I thought such a project would be so easy. Turned out I was wrong. For every software application development, there will be a few tricky corners that all developers cannot foresee. When they finally encounter such a corner, they would not know what to do, and will make quick but not-so-wise decisions. When all these hastily made decision accumulates, the end product would become a pile of crap. It is workable, but will not be user-friendly. This is especially true when one underestimate the project, which is what I did. This explains why I had to write the same application three times.
I started the development of the third version in Dec 2017. I switched from GitHub to BitBucket, because BitBucket have already started providing private repository for small team. And GitHub only started allow private repository for free in April 2020.
For UI of administrative site, I chose AngularJS. At the time, I have used AngularJS for 3 years, and felt very comfortable with it. The web server piece I used Spring Boot. It turned out to be a great decision. There is no need to worry about application container. It is easily deployed. And starts up within 9 to 13 seconds, super fast. The use of AngularJS really made a difference. The UI interaction written in AngularJS was much more organized and clean than the code design with JQuery. I was very happy how it turned out.
For the public blog site, there is no need of a powerful JavaScript framework like AngularJS. The blog site's main objective is to make information available to a wide audience. To me, fancy UI interactions serves no real purpose (other than catching people's eyes), and it would add longer development time, testing time, and worst unnecessary complexity to the blog site. Hence, I decided it is best to make it simple, as static as possible, with minimum UI interactions. I didn't come up with this conclusion myself. Many others have also made similar comments. I took their advice and kept the blog site as simple as possible.
Besides the two web sites, I created three more projects, one for the data access, one for services layer, and one for data models and entities. That is five different different projects. This is a departure from the previous versions where there is just one project with all these different things packaged together, a monolithic project. The down side with this splitting strategy is that some how it took a long time for everything to be built. It was probably an OS thing. I was compiling it on Linux, and it was lightning fast. Another thing I find it annoying is that I cannot do debugging with Eclipse while running Spring Boot. Other than these two, I was quite happy with the decision of splitting the monolithic project into multiple projects.
For this latest version I was determined to make it the best (as best as I can do with limited time and resources). I took some of the best things I have discovered, and put it in this project. For example, I like the idea of splitting the HTML UI elements on a page so that some of these can be reused. So I used the taglib with the JSP pages. It made the page markup almost unrecognizable, yet the approach made the static UI components reusable.
One cool thing I have discovered was that all the site objects had shared some common properties, like title, keywords, category, published/unpublished, description, So every object has a metadata super class which encapsulate all these common properties. Then each has its own set of unique properties. The cool thing about this is that all the queries that work with these objects are common. So I had packed all these queries into a single class, a query builder. This way, I avoided a lot of duplicated codes. In many cases, these reusable methods can be re-arranged and fulfill a new functionality. This latter situation is sometimes a pleasant surprise.
Another challenge I found was the design of a functionality for list and pagination. No matter how hard I tried, I was not able to eliminate the code duplication. In the end, I decided not to eliminate the code duplication. I rewrote many parts of it, and made all of the duplicated code exactly the same. This would make the maintenance a lot easier.
I did a lot of rewrite, It is unlike the previous version, where I wrote the functionality and moved on. Most of the times, rewrite is absolutely necessary. The first try of a design is almost always bad. It is necessary to redo the bad stuff with a different approach. You see, rewrite an application system from scratch is very expensive in terms of time. So it is best to redo only parts of the application at a time, and continue with the application. This is the best approach. Unfortunately, it could a long while to get an application completed.
This leads to the last point I like to discuss. The product does not have to completed when it is deployed for use. When the product is in a good state, maybe it can be deployed for use. Then it can be improved, then redeployed to use. I discovered this a little too late. If I adapted this approach a little earlier, I could have made this application out in the public a lot earlier. After I discovered the deployment was really easy, I realized it might be possible to have this application to go with such iterative process, release small and release frequently. For the next product, I will try iterative development more.
Towards the end, one big challenge I faced was the upgrade. When I started this, I was using Java 1.8, and Spring Boot 2.1.1.RELEASE, When time comes that I needed to release this. I decided to use a better version of Java. So I went with OpenJDK 14. The maven build worked as expected, But there were a bunch of warnings. Apparently, there were big changes in Java 9 that caused the warnings. I was not able to fix it then. As of today (6/19/2020), I finally fixed all these warnings. There were another runtime warning about Illegal Access. It was from the JSTL/taglib jar. Not sure what to do with this. The problem with today's technologies, they move fast and it breaks backward compatibility. I will let this one go for now. Man, I learned so much just by fixing all these little issues.
This is a good place to stop. The last section here I will summarize some of the most important things I have learned. There is no way for me to do something and get it right the first time. So it would take me at least two three times to redo and do it right. It is important for me to do a project first time, learned the mistake and correct in the subsequent ones. This is the only way to create great product.
One important thing I did learn from this is that I should release product out as soon as possible, use the product, find issues, then fix, and release again. It took way too long (18 months) for me to get the project into a final staging to release. In the process, I had to make trade offs, and things were never out of the door. For this project and the next one, I will do frequent releases. As soon as a product has a good shape, it will be released into the general public. I will try it out as best as I could and fix all the issues, then release again. This is the future. I should embrace it.
For a typical blog site, there are only a few concepts needed t make it strong and powerful. One is the site objects like the pages, posts, and updates. Pages and posts are the same things, we can distinguish then by type. The site update is similar to the pages or posts, but much simplified. Site resources (specially with the file resources), they are all the same, distinguished by types. Each resource has a metadata file associated, we load and send the resource by look up this metadata file first before going into the database, this could be quicker. Site users, and links are two different things from the post/page/updates, or the resources. Comments associated with the pages or the posts, Private messages are by itself. Both of them share the same common properties. All of these are lists of objects, and needs pagination. On the admin age, all objects are loaded at once, and the pagination are done in memory. The blog sites are loading site objects by pages. These are the basic overview of the technical details of this project.
This post is long enough, I will end it here. This is not the best post I have. I will be improving for the next one.
There is no comments to this post/article.
