Site Title

Tag: technology

  • From Operations to Architecture: Leveraging Microsoft CoPilot to master TIBCO EMS and FTL

    I have a strong belief that Information Technology Operations is one of the most critical departments of an organization. In a large and multifarious organization like the Canadian Tire Corporation, Information and the systems that this information flows through act as a nervous system of a company. These heterogeneous systems comprise a complex umbrella of various technologies and products. It is the job of an I.T operations team to ensure that these systems work seamlessly, and are always operational. It is no exaggeration to say that the operations team guarantees customers receive continuous value from the services provided by our firm.

    Within this vast and intricate world of I.T Operations, I belong to a team that is responsible for making sure that the TIBCO infrastructure runs smoothly and providing quick resolution of production issues that may impact our real-time services. Resolution of these issues require a deep understanding of the underlying systems that constitute our TIBCO environment. It also requires a holistic understanding of various other technology products and areas that comprise an integrated I.T ecosystem.

    Drawing from over a decade of experience working in TIBCO operations at various organizations, I have come to form a conclusive opinion that the field of operations is tightly integrated with the architectural design on top of which the operations run. To support and troubleshoot the operational problems, it is imperative to learn and be thorough with how the underlying system is structured. In this regard, I would like to present my case as to how Co-Pilot has played a transformative role in bridging the gap of my understanding between TIBCO Operations and TIBCO Architecture. It has fundamentally changed the way I approach the complex incidents that I encounter in managing and maintaining this Middleware infrastructure of which TIBCO is a component.

    Within the broader net of TIBCO operations and TIBCO Architecture, I would like to put forward two interconnected examples that demonstrate a consolidated view of how Microsoft CoPilot acted as an enabler in transforming my comprehension and implementation of day to day work here at DiT team within Canadian Tire.

    Case Study 1 – Understanding the new TIBCO EMS and TIBCO FTL design: Enterprise Messaging Service (EMS) is the messaging backbone of our environment which enables applications to communicate with each other via XML-based messages. These XML based messages contain of the actual payload or business data that the applications process. This workflow is directly linked to the customer-facing services provided across various Canadian Tire Corporation (CTC) channels. The EMS server utilizes destinations like queues and topics to manage messages generated by producer applications and retrieved by consumer applications (primarily BusinessWorks or BW applications in our environment). In our legacy systems, we have been using a standalone EMS system in our production environment running on version 8.4. This legacy configuration does not employ an Active-Standby architectural design, which would otherwise ensure that if the primary EMS crashes, a standby server takes over with minimal to no disruption.

    The design/architecture of EMS / FTL drastically revamped when our entire TIBCO based ecosystem was migrated to a newer version. This was a major shift in building a new set up with both TIBCO BW migrating from the older 6.5 version to the newer 6.10 version and EMS migrating from the older 8.4 version to the newer 10.3 version. The major shift in the set up was the use of TIBCO FTL , acronym of Faster than light, as the foundational backbone of the new EMS. The new setup is called EMS on FTL stores, where FTL provides a synchronization and communication mechanism between the three EMS services that form a Quorum in the new design. Since the architecture was setup by the external HCL team, it seemed an uphill (but an imperative) journey to understand this complicated system design that acts as a bedrock of our operations. This is where CoPilot came to the rescue.

    I regard AI in general, and our CTC based CoPilot as a modern alternative of a book club or discussion forum serving as a space to brainstorm ideas and refine conceptual understanding. It is like a laboratory where one can “stress-test” his ideas. To master the new EMS and FTL setup, I adopted a conversational learning approach, which is central to how Copilot builds expertise. I got to know how different layers of FTL , consisting of FTL cluster service, FTL persistence service and FTL realm service work together as a cohesive system to harmonize the workings of the EMS cluster. This intensive, two-way dialogue was highly iterative: Copilot corrected gaps in my understanding of the new design, while I occasionally corrected its contradictory statements. This collaborative process ultimately allowed me to build a crystal-clear insight into our new EMS on FTL architecture.

    The grasp of conceptual understanding did not occur instantly. It required several back and forth arguments. As mentioned above, building on knowledge with the use of CoPilot is an iterative process. This back-and-forth approach helped me connect the dots, moving from an exhaustive grasp of how the FTL cluster server operates as the topmost layer to understanding how the FTL persistence service (used to persist data in the EMS cluster data stores) and the FTL realm service (used to synchronize and replicate configurations across the EMS servers) work together beneath that layered structure.

    The first test done with CoPilot was to check how data replication happens across the EMS cluster hosted on three Linux based servers. To kick off the process, I provided CoPilot with the configuration of our EMS cluster system by attaching the Business-EMS.yaml file which consisted of two major pieces of information. The first was the path of the data store where all the persistent messages are stored and the second was the path of rs.db file which includes the configuration files in a .JSON format. In order to check if we have real time data replication, CoPilot recommended that I verify if the data store path is on nfs (Network file system) or XFS (which would indicate a standalone data store). I ran the df -Th /path-to-store-File and the output showed that the file system is XFS rather than a networked file system.

    I was not convinced thus far since the store.db file in the store directory was almost the same size across all the three EMS servers. Upon putting forward this point, CoPilot prompted me to do a hands on test by suggesting that I create a test queue and push a message to this queue on the Active EMS server, kill the Active server so that Standy EMS Server takes on the Active role and verify if the queue and message appeared on the new Active server. I executed this test and observed that the queue indeed did got reflected on the new Active EMS server but the message did not got replicated. The concern was still not addressed. If the data replication in the data store was not real- time, how did the sizes of the files in the data store across the three EMS servers hosting the cluster matched?

    Prompting this concern to CoPilot, and engaging in more intense conversations, I got to know that only persistent messages (these are the messages that get written to the disk so that they are never lost , unlike the non-persistent messages which get stored in EMS memory) get replicated real-time. So, based on our “discourse”, I performed another test of similar nature and pushed a persistent message on the test queue and I was glad to know that it was reflected on the new Active server. Knowing this aspect of the new EMS/FTL architectural design was the first big step towards building a solid foundation.

    The next vital step was to understand how the EMS server interacts with FTL during boots-up. From my interactions with generative AI, I have experienced that the best way to extract the most from an AI agent like CoPilot is not to just ask a straight question like “how does something works?”. Instead, a better approach is to prompt it with your own initial conceptual understanding and build upon that foundation based on Copilot’s responses. This makes the interaction far more engaging and helps sharpen your knowledge within a specific context. I believe this collaborative feedback loop is exactly what makes prompt engineering so powerful.

    I followed the same strategy and after some oscillating conversations, I was able to refine my understanding about the EMS start up/boot up process. Over a period of time, I have developed a habit of presenting a final summary to Co-Pilot based on our long conversations to chisel out the rough edges and close the loop. With the problem at hand, I did the same and proffered my final summary with the process that looked like the following – “When EMS server starts, it brings up its main process called tibemsd which calls the FTL realm service, which in turn loads the configurations from rs.db to the memory of tibemsd process. When we create a queue, tibemsd process stores it in its memory and also connects with FTL realm service which in turn stores it in the rs.db file across all the three EMS servers.”

    The third step was to know how the FTL manages the Fault-Tolerance of the EMS. To know about it, it was imperative to first be acquainted with the difference between FTL cluster service, FTL persistence service and the FTL realm service. CoPilot did a wonderful job by presenting a diagram of how these three FTL processes are layered and function in conjunction with each other. This was the perfect example of the quote that “A picture is worth a thousand words”. Through the diagram, I got to know the complex idea in a very efficient manner.

    Case Study 2 – Troubleshooting EMS / FTL issue: Knowledge brings confidence but like any area of research, this confidence is tested when you encounter the problem in real-time. In my case, this “opportunity” came when our team got alerted that our Active EMS in production suddenly and unexpectedly crashed and the Standby resumed the role of Active. Drawing on my understanding, I was able to analyze the logs of the crashed EMS server in a more effective manner. The logs indicated a “temporary network disconnection”. So we assumed that was a one-off incident and might not occur again. However, the issue repeated itself after a couple of days. I knew it was something beyond just a minor network disruption. I contacted the CTC Network team to know if there was any change made by them on our EMS servers during the timeframe when the issue occurred. Network team responded that there was no activity carried out on their end.

    CoPilot once again became my “trusted colleague”. After I described the nature of the “roadblock” we were facing, It suggested that we need to upgrade the EMS to version 10.4.1 from 10.4.0 and FTL to version 7.1 from 7.0. By scanning the vast ocean of TIBCO documentation, CoPilot highlighted an inherent and a known bug in the current versions of the software that we were using. With management’s approval, I initiated the process to implement an in-place upgrade of the EMS and FTL instances. An in-place upgrade allows you to deploy the new software using the existing configurations, meaning we did not need new servers to execute this overhaul. Not only CoPilot described the entire process of upgradation, but also facilitated in deepening the comprehension of various features of the new versions. It laid out the exact plan of action including any prerequisites and migration stages required for this upgradation.

    With the enriching assistance of CoPilot, I was able to upgrade EMS and FTL to the newer versions in our development environment, and we are now ready to execute the process in the higher environments.

    To sum up the above two inter-related cases studies revolving around TIBCO EMS and FTL, I would like to mention that the value that you get from CoPilot is directly dependent on the input that you provide. It ultimately boils down to “prompt engineering” which includes scraping the concepts through long and concentrated conversations till every dot connects. This iterative approach is exactly how Copilot transforms your outlook and method for handling daily technical work.

  • Why Gandhi would have championed Open Source Software?

    One of the greatest polymath and genius of all times, Albert Einstein, once said about the political views of Mohandas Karamchand Gandhi, called as Mahatma Gandhi by his millions of followers, that “I believe that Gandhi’s views were the most enlightened of all the political men in our time.”. Going over his autobiography, “The story of my experiments with truth”, I find Gandhi as one of the most inspiring personalities that I have read about. As someone who works in the Information Technology sector with a keen interest in Open source in general and Linux Operating system in particular, I could vividly draw parallels between the ideology of this great political and intellectual Giant and the ideology that powers the Open source methodology.

    I have come to a conclusive opinion that had Gandhi lived today or during the times of technology and software boom, he would have become a “brand ambassador” and a thorough supporter of the open source movement. The principles that this “saintly leader having a global impact” followed and practiced in his life have several parallels with the foundations on which the open source was built. These ideals continue to stimulate and motivate millions of people around the world. Here are three resemblances between the modus operandi of Gandhi and open source that in my view reverberates with everlasting impact in the world we live in:

    A Community driven approach : Gandhi, during his days in South Africa was deeply inspired by Ruskin’s book “Unto This Last“, which he read and put into practice by establishing the “Phoenix Settlement”. Later, he was inspired by Leo Tolstoy’s work “The Kingdom of God Is Within You” and established “Tolstoy Farm” near Johannesburg. The common aspect of both these establishments were that they were community driven places, an approach also central to the open source developmental framework. Communities form the nucleus of all the software written in the open source world. People who were members of the aforementioned communities formulated by Gandhi participated with a desire fueled by an intention of embedding a novel approach to life while making the society a more participative and responsible place to dwell. Participation is key in the open source world as well. Both these systems transmogrified the basis of the human thought.

    A Do-It-Yourself (DIY) approach to life: The world of open source software is a world of experimentation. It is a world where the concept of DIY is implemented to its fullest and brought into its fruition. Engineers, hackers and software enthusiasts develop products that millions of people can play with, modify and recrate completely new innovations out of those products. Zealots can follow easily accessible in depth documentation to “break and make things”. This creates an enterprising environment benefitting technology, the business and the society as a whole. As I read his autobiography, I see the actions of Gandhi imbibing the same DIY scruples as the one’s described above. The “spiritual giant” was adept in experimentation and trying out new concepts himself and energizing people around him to do the same. Whether it be practicing as a Lawyer, working as a public servant, his explorations in dietary (especially his inclination towards earth and water treatment) or even learning geometry, Gandhi took the existing knowledge, mastered it, and inspired the world to implement the same in their lifestyles. It also implies the broad applicability of ideals and principles that forms the backbone of the open source realm.

    Power to the people : Through Free Software Foundation, the software pioneer Richard Stallman advocates to distribute the software in such a way that it is free for people to use and modify its source code. In 1989, he co-founded “League for Programming Freedom“, through which he campaigns against the software patents and restrictions on the use of “digital software products”. In a nutshell, he is a proponent of giving power to the people. On the same platform, Gandhi’s whole life was dedicated to empowering those who were deprived of exercising their rights. The core of both these ideologies was to challenge the existing norms and protocols and bring about a “constructive revolutionary change” in the lives of people and the world we live in. Evolving and nurturing the expression of individuality forms the spinal cord of both the philosophies.

    Whether it was the relentless pursue of truth which Gandhi immersed with the struggle for independence during colonial era or the software upheaval and revolution brought by the open source methodology to create a better technological ecosystem, I see deep affinity in both these approaches to tackle the pressing problems of the world from which we all can learn.

  • How entrepreneurial are the Indian educational institutes?

    Waterloo is a small University town located in the Canadian province of Ontario. With a population of less than a hundred thousand, it is an entrepreneurial powerhouse. With more than one thousand start ups credited to its innovative history, the region is one of the most progressive startup ecosystems in the world. Situated twelve thousand kilometers away is Mumbai, India’s financial center. Indian Institute of Technology, or IIT located in this financial hub is one of the primary educational institutes in the Asian region. Its E-cell has spawned more than two thousand companies, making it the largest in the Indian subcontinent.

    This is not just a comparison between two of the finest institutes, it is a comparison between a highly developed G8 country and a one of the fastest developing countries in the world. Although, India has made great strides in developing an entrepreneurial environment, it still is a land of extremes. On the one side, we have universities that rank among the best in the global rankings, but on the other hand, we still have a significant population who can hardly put a signature with a pen.

    Indian startup scene is becoming more and more inclusive over the time. With an active role played by the governmental sector, the outreach program is reaching and changing lives of millions of people across the country. A few days back, I came across an article on Gatesnotes that talked about DroneDidis, an innovative public sector scheme which empowers women in rural Bihar to use the technology of drones to increase their agricultural output by effective use of fertilizers. This exemplifies how Indian entrepreneurial landscape is getting transformed, even though at a gradual speed.

    To create an enriching and a supportive platform for the ideas to nurture and mature into successful enterprises, it is not only the government sector that has to chip in, it is also the private players and other organizational stakeholders that need to contribute in the creation of a dynamic start up society. In my view, the Indian economical and business sector has gone through four waves of evolution. The first wave was the opening up of call centers after the liberalization of the Indian economy. The second wave was the onset of Information technology industry. The third was the investment of United States into the research and development sector in India, primarily in the cities of Hyderabad, Bengaluru and Pune. The fourth ongoing wave is that of the boom in startups or technology oriented small scale businesses.

    I firmly believe that educational institutes can play an instrumental role in developing and nurturing the entrepreneurial energies of the Indian youth. They can act as a “beacon of inspiration” and provide the much needed mentorship, especially the in nascent years of the budding entrepreneurs. Although more and more universities and educational institutes are embracing innovation and entrepreneurship into their academic fabric, much still needs to be done. While I do agree that we have come a long way as an innovative and a productive society, there are challenges ahead of us.

    Recently, I was going through the specializations offered by the Master of Business Administration , or MBA program of my alma mater and I was surprised to certain extent that entrepreneurship was not one of those. It is not only my university business school that still hasn’t introduced entrepreneurship as a stream, the story is same for majority of the educational institutes across the country. The startup scene on the campus is still concentrated in the premier universities of the country like the IIT’s, IIM’s and private educational colleges.

    I think this is where we still have a gap and this is what needs a change. While I do agree that academic and vocational institutions do need to create a “white collar” workforce and prepare its pupils to excel in corporate jobs, but I strongly think that universities need to invest its resources in development of “Job creators” as well. Incubators should be set up on university campuses which not only nourishes students in its technology departments but also encourages young and aspiring minds from arts and commerce departments as well. Students should have an access to mentorship, to investment capital and access to the market where they can launch their idea. Cutting edge paradigms and business concepts like Design thinking should be absorbed in the entrepreneurial training and skill development. The start up growth has to be inclusive for India to compete with other countries.

    It is rightly said that the development comes when the highest level of technology reaches the lowest level of the society. The “startup culture” needs to percolate deep into the Indian society. Our educational institutes have the potential to accelerate this development and catapult the nation to the global stage.