Deconstructing Cloud Computing: My Experience
For startups, some type of projects come once in a while, especially a foray into areas you are not familiar with. An opportunity came for me to pitch a portal to a university. Proposal was quickly packaged — for many of us who have submitted 10s of such proposals, it’s a basic stuff — and sent. The IT department called for a demo. As a Java programmer, something close to a system architect, I put together an enterprise resource planning (ERP) system for a company which replaced their Sage accounting software. In fact, the enterprise system does more than accounting with 5 modules (Sales, Warehouse/Store inventory, Accounting, Expenses, Administration). I built the system from scratch for a year plus, using Java enterprise technologies, MVC programming model, Bootstrap, HTML5 and it was deployed in a hosted private Tomcat over MySQL.
Now the potential project at hand needed a similar system. However, the institution needs to get up and running quickly. Not having a university portal in 2018 is awkward. One’s mind will wander wide wondering which of the higher institutions in Nigeria hasn’t implemented a university portal. Almost all have, those that haven’t are mostly private universities, some not more than 10 years old. Most public universities have, the norm is to change vendors managing their portal once a new sheriff (VC) is appointed. So for this private university, they have tried to use internal staff expertise to get up and running, but building such a system requires an ICT company with experience in software development at an advanced level.
After a meeting with the IT staff, I went home with my team (we are just about three) but as the project lead, conceptualizing the system was my forte. The ERP I mentioned earlier was conceptualized by me after I had tried to get a colleague who now works with Crossover to put it together. In those days (before 2015), we didn’t have much knowledge. We wanted to deploy with Spring Framework but installing the framework proved abortive and we had to take the route for building from scratch.With Bootstrap to serve as our views, we set out developing the models and controllers and over the next one year, the system was up and running.
On this current project, one thing I have learned as programmer is not to reinvent the wheel. Java language being my major is an open source platform so I am an open source evangelist. I went online to search for open source university management information system (UMIS). I found one, Open Baraza and when I searched further, it has been around for a while (before 2015), was developed with programming models I was familiar with, and it had the GNU General Public License (GPL) which “is a widely used free software license, which guarantees end users the freedom to run, study, share and modify the software.” I downloaded the platform to run on my computer system. First, it runs on PostgreSQL, being familiar with relational databases like Oracle and MySQL, this was nothing extraordinary. I installed the database on my Mac and ran the application and it didn’t take me up to a full day to have it running. I was relieved. I went for the demo and basically try to convince the staff that this platform which is what Babcock uses is robust. I had examined the source code and was confident I could manipulate anything in it. On the day of the demo, the technical head of ICT who I have been speaking to who was favorable to using Open Baraza was’t around and his team which include the university head of ICT was skeptical. The programmer staff, also a lecturer was stentorian about having a live version as proof of concept as they all have Open Baraza on their system but I guess getting it online and live was the issue. I later discovered it. Tweaking Open Baraza UMIS platform was not a yeoman’s job.
So I was challenged to present a proof of concept which they can access like a live application and that was the beginning of the daunting task. I like programming challenges. I trust my prowess when it comes to developing any system in Java. So I went home, I had to understand that system and deploy it online. I had a Tomcat running for the ERP so the best bet was to package the program, deploy in the container and we are good to go. But no, it won’t work that way because the person(s) who developed this system, was an expert at standalone application, in fact, the platform has an embedded Tomcat and programmatically the system runs highly customised. You cannot package it as a WAR (web archive) file for deployment. Folders are arranged in an unconventional way and if you try to open create a project from it in an IDE like Netbeans, you are sure to have your IDE mess things up. It will successfully create but fraught with errors but your WAR file cannot run.
Here was where I started brainstorming. I navigated from folders to folders, reading all the documentations and README to understand this platform first. With the embedded Tomcat, you can run it on localhost, that’s all you can achieve but when dropped in Tomcat, online or offline, logs tell you the UMIS can’t run.
The idea of virtual private hosting (VPS), something I hadn’t used before came to my mind. I know Linux OS to advanced level and I was like, get this folder up in a VPS and run the script and boom, we replicate what I have on my localhost. It is easier said than done. I went online, got a VPS from Web4Africa and another one from WhoGoHost. I opted for the later because after purchasing from Web4Africa, activating the service was taking time, I was impatient. Being a premium customer of WhoGoHost, I had to order from them but I was skeptical about such a service from Nigeria and preferred Web4Africa from South Africa. However, WhoGohost proved to be more responsive and that is where the application is currently deployed.
Here was where I discovered Cloud Computing and Linux DevOps VPS in practice. The first task was to install PostgreSQL. I have installed 10s of databases and done so on my system, but getting this to run on the Linux CenOS 7 was not straightforward but I got that sorted. I registered a domain and the technical guys at WhoGoHost got it pointed to my VPS IP address. The next task was to install a web server, I contacted the guys at WhoGoHost, they reasoned that I needed to buy CPanel. C’mon I had spend almost N40,000 and this is just a demo. This was day 2 of my marathon hacking. I succeeded in installing the Apache Server 2 without the technical guys. Surfing Google to find answers to each issue I encountered was my usual approach. The newest website I found was digitalocean.com, I was very familiar with others where you find most answers specifically serverfault.com and stackoverflow.com.
After installing the database and Apache, I deployed the UMIS folder to my home directory on the VPS. The first hurdle was getting the SWING desktop application which is used in populating the database to run. Opening desktop apps from a VPS requires allowing the VPS server and system to communicate. After many hours that was resolved by activating xhost. This was the point the real work started. The program ran on the server but getting Apache Server to communicate with the embedded Tomcat to echo on the domain was the issue. I tried all sort of fixes thrown at me from Google. With Linux OS, there’s no straightforward way of accomplishing one task. With the different flavours of OS, different versions, you have to find the command suitable for your OS from those who have encountered such issue. This was where I had to switch to the source code version instead of the runnable version. I had to prove to myself I actually understood Java and have to customise the Java program classes to get this portal to run. I understood that before my Tomcat could talk to the Apache Server, I needed a proxy. The simplest it could use was Mod_proxy. I went about configuring it. This was the second day. My people, it didn’t work. By this third day, I was tensed, it was Saturday evening, this demo should be up by Monday, I have to prove to these guys that I know my onions. I was like in a trance. Anyone who called me, those I could pick their calls noticed the uneasiness and tension in my voice.
After some hacking, configurations, brute force (trial and error), I resolved to use Mod_jk which was a more sophisticated server connector module. I succeeded in installing it in a short time because by Saturday, I had learned so much about Linux DevOps that I was feeling fly. My experience with bottom-plate coding, reading and disambiguating server configurations came in handy. I am this sort of guy that reads server documentations like a textbook. I play around with Tomcat, Glassfish and JBoss server configurations. I study documentations like I was going to write an exam while coding. This was Sunday evening and I had the application running on the VPS on embedded Tomcat and Apache Server running and Mod_jk installed but I can’t see the application on the domain in a browser. The servers can’t communicate. In the process of installing the connector modules, I had extended the source code and added two connectors to the Tomcat class, normal HTTP1.1 and AJP.
What I forgot was that I needed to recompile the project and redeploy to have the changes I made propagate. I thought about this after reading the logs for Mod_jk and discovered it couldn’t link to the Tomcat port and Tomcat wasn’t even listening to that 8009 port. This was Monday already. I was in distress already. I CANNOT fail. I must deliver. This led me to check the jar file of embedded Tomcat java class and I discovered the connector methods I added wasn’t reflected since Saturday. This had made me get all the configuration on the VPS server correct as I thought the issue was from the server.
But once that thought of checking the jar files I deployed came and I found the disparity and I was like ‘eureka, I found it.’ I had been binge drinking all these five days, recall I hadn’t left my room these since Saturday because I went to play football Saturday morning, which didn’t take place because the pitch was flooded. So I five days I hadn’t left the house. So I installed ANT compiler, looked at the build script, made changes as necessary and compiled and uploaded, just like a smooth operator, the portal echoed on the browser. I poured myself a shot, walked out of my with all the tension in my head subsiding and fizzling out. I could breathe. I sent some Whatsapp messages to the university IT guy, ‘the portal is up’ and some links. Had a few boisterous phone calls and conversations with my guy at home who is part of the team but mostly working on the UI and another colleague. That was not the end of it. The portal could only run if I SSHed the VPS. Once I log off, or network cut, it went down.
By Tuesday, I needed to get it running as a service. At this point, I was taking things easy because I had already done the difficult part. After setting up the startup script as a service and it kept failing, reading the logs informed me the service couldn’t start because the embedded Tomcat couldn’t find the folders. I had to provide absolute paths by tweaking the embedded Tomcat java class replace the getCurrentDir method. I recompiled and deployed, it was running as service, 24/7. At this point, I was feeling myself. I know if the guys at the university could, they could have done what I did a very long time ago and had portal running but my experience of cloud computing and Linux DevOps VPS requires some above intermediate level of software development.
You need to be an expert and resilient to do what I have done without assistance from anyone. But this is how I learned programming. Aside the Oracle Database administration program training I was taught years back, everything I know, I learned by reading and practice. I taught myself, just like I have done as described here in a production environment and have learned what many would pay thousands of dollars and take months to learn. If you want to learn, contact me. I’m willing to share the knowledge. I’m OUT. Stand up for the champion!