Fig. 3. Illustration of Lojika’s Deployment Pipeline Fig. 3. Illustration of Lojika’s Deployment Pipeline

Coping With Challenges

Lojika also known as ”Lojika Field Labs” is an R&D company which has more than enough national and EU granted innovation projects. By its nature, research and development process of innovation projects contains so many ambiguities. Lojika is a company which gathers information directly from different fields around the world for its innovation projects. Due to the fact that Lojika is trying to solve the society related issues on the subject of car-pooling, Physical Internet, sharing economy and crowd-sourcing, embracing changes becomes inevitable necessity for the company culture to succeed. Some of the challenges we have faced are:

  • Clarification of ambiguities regarding the needs of customers
  • Trying to adapt to rapidly changing environment
  • Fast evaluation and implementation of the field feedback for fast responding
  • Minimizing analyze, develop, test and deploy cycle delay
  • Increasing participation in all phases from the beginning of evaluation the feedback to the release time
  • To ensure transparency

As to success in changing environments, selecting a proper software development approach becomes a key factor to adopt changes quickly, deliver new idea and change requests without less doubt and efforts. Right at this moment, continuous delivery practices come into play. Continuous delivery makes it possible to continuously adapt software in line with user feedback, shifts in the market and changes of business strategy. Testing, support, development and operations work together as one delivery team to automate and streamline the build, test and release process [15].

As it is illustrated in figure 2, Lojika’s communication flow is composed of three main processes. Field study is the best step to understand users and identify their needs additionally new ideas emerge during this process, which marketing team leads besides product and development team are the important part of it. Analyzing the Feedback is another main process of the communication flow which product team is the responsible for distilling the gathered information from the field. The last step of the communication flow and the subject of this study, Delivery Pipeline, which is the baking step of the whole flow. That means development is just a part of the continuous delivery. In this process, information coming from the previous step goes into the baking pipeline. That is to say that baking pipeline symbolizes development or build pipeline which is the most significant part of the continuous delivery. Some feedback are implemented in a build which is also a release candidate; therefore, ready for deployment to production. This is the heart of continuous delivery and eventually this approach makes the software always ready for every build baked in the deployment pipeline. To sum up, continuous delivery is a vital part of building up a continuous communication cycle between stakeholders including users in the field.

Taking into consideration of these which are given above regarding Lojika’s culture and challenges, a tailored version of a deployment pipeline is described in the next section.

Building the Deployment Pipeline

The deployment or build pipeline is a high level visualization of connected jobs which compose the entire pipeline. All separated jobs have their own responsibilities and they are connected to each other as upstream and downstream which means that each job can trigger its downstream jobs and transfer any information to them based on its success or fail. Considering the fundamental principles and practices, with the use of some tools and technologies, it is possible to build and operate a successful deployment pipeline

Creating a delivery workflow for the company is more important than choosing tools and technology. Nevertheless, technologies and tools used for the pipeline are described in table I along with their explanations.

As shown in Figure 3, each step is numbered one by one. In
the next part of the study, each working phase in the workflow defined for Lojika’s TAG project is explained in detail. Along with the difficulties encountered, the technologies and tools used to accomplish each step and the steps taken to solve them are explained.

A. Development Phase

Everything starts after feedback is received. Once the analysis work on the feedback is completed, it is transferred to the development phase. Developers complete their work in their own development environment and then submit them for the review. If the review is positive, it is merged into the develop branch. It is now ready to be sent to the continuous integration server. Developers collaborate with Git and the source code is evaluated through Github.

#A.2 is responsible for transition from development to continuous integration. At this stage, changes to Github are
pulled by Jenkins, one of the most well-known continuous
integration applications. Task #A.2 is also the first task to run
the deployment pipeline.

B. Continuous Integration Phase

#B.1: Building process of the last version of source code starts. If the compilation process fails, then developers are informed about this failure using Jenkins email plugin. Otherwise pipeline goes on.

#B.2: Run automated unit and component tests in the same job. If any of the tests fail, developers are notified by e-mail. If there is no problem, the next process starts.

#B.3: SonarQube is used for quality measurements of source code. With Jenkins’ SonarQube integration, source code is made necessary for quality measurements. If the quality values are below the specified threshold values, the process will not continue and the developers will be informed.

#B.4: Artifacts (Jar files, metadata etc.) created for later use in the case where everything is complete and error-free is uploaded to the JFrog artifact repository.

#B.5: Both low-level documentation (JavaDoc etc.) and REST API documentation required for client developers are automatically generated. Swagger is preferred to create REST API documentation.

#B.6: Lojika’s development architecture is multi-layered. That is, both the infrastructure and the frontend development
teams are positioned separately. The client team that needs to
work with the infrastructure to be able to access these services
with minimum effort so that they can run smoothly. In this step,
a Docker image is created that the client developers can use
in their local development environments. At this point, client
developers can pull the desired version of the backend services
to their machine at any time.

#B.7: After each step in this process is successfully completed, both backend and client developers will be informed.

#B.8: The same build is used for each phase transition. This means that if you want to design a successful pipeline,
as mentioned in the previous chapters, the ”Only Build Your
Binaries Once” rule must be followed. So, the artifact created
in step B.1 itself is transferred by the trigger in step B.8 to
the next ”Testing Phase”.

C. Testing Phase

Some steps are progressing automatically in testing phase, but in some cases manual intervention is required. This stage can be performed completely automatically or fully manual. But at the end of the day each build that comes over the pipeline must be approved by a QA role and decided that it is ready for the next phase.

#C.1: The build on the pipeline is deployed on the test servers. This is done with plugins provided by Jenkins.

#C.2: At this time, all kinds of database changes are applied using Liquibase. Managing database versioning and changes is one of our main challenges. We have also automated the management of database changes, such as automating all the work that will disrupt the pipeline and cause waste of time.

#C.3: Writing tests for every case and each scenario is a very ingenious business. In general, tests are written from the most important and priority scenarios to the most insignificant ones. It is a fact that time can not be allocated to risky and important tests due to the time spent for unnecessary tests. Therefore, automating tests that are relatively easy to automate will be strategically more reasonable. This is also Lojika’s strategy. Scenarios that are easy to automate and do not have a lot of integration with external systems are run by Appium for both iOS and Android clients. On this side, more time can be allocated for risky and dependent scenarios.

#C.4: The risks, uncertainties and challenges in this step are very different to the previous one. Non-functional tests, however, need to be fully automated to ensure continuity and sustainability. In particular, the performance, load and stress tests of the technically risky parts, which may be overloaded by the users, are automatically performed by Jmeter and Blazemeter.

#C.5: As already mentioned, it is absolutely necessary to be approved by QA so that this phase can be successfully completed. Thanks to the Jenkins promotion plugin, the person who is responsible for promotion promotes the current build to the next stage of the pipeline.

D. Release Phase

#D.1: The same artifact that completes each stage successfully is ready to go to production-like environment this time. The purpose of this step is to create a simulation of the real environment using a production-like environment before the release.

#D.2: The same operation as in C.2 is applied.

#D.3: Some automation again. Although the artifact remains unchanged and everything is positive during the testing phase, it is a good idea to automatically test specific scenarios the last time.

#D.4: Continuous delivery without manual intervention is not possible. The good thing is that an artifact that comes up
to this stage is ready to release. Without any rule, everyone
involved uses this last build with realistic scenarios for the last time before the release.

#D.5: There is no rule that every artifact should be released. According to the pipeline logic, every promoted artifact can be released at any time. Depending on the release strategy and conditions, the release decision may be delayed or an old artifact may be chosen to release. It is to be remembered that with continuous delivery, each artifact is expected to be a ”release candidate”. But whatever it is, there must be a role to make a release decision anyway.

#D.6: After the promotion decision, the artifact is deployed to the production environment and stakeholders related to this process are informed about the changes.


In Lojika, continuous delivery has been successfully implemented and 2070 builds have been built up to now. 827 of them have been successfully uploaded to test servers. 229 of them were uploaded to the Staging server. 101 of them passed through production.

Until now, unit and component tests have been run more than 620,000 times on average.

In addition to numerical improvements, cultural and organizational improvements have been observed as well;

  • Quick response to user feedback
  • Improved communication between both product and software development teams from start to end
  • Increase in company-wide participation
  • Reduction of hot-fixes
  • Ease of managing database changes in any environment
  • Improved software quality

Conclusion and Future Work

This study attempts to explain the benefits of continuous delivery, which is part of agile discipline, on a case study. We discussed that the company can successfully implement a continuous delivery workflow in line with its own needs and improve product and software development processes at this point.

Obviously, this study promises that continuous delivery practices can be applied not only to large institutions, but also to startups.

Moreover, some steps need to be improved in relation to the workflow in figure 3. In particular, non-functional tests, for instance, stress and security tests needs to be improved. The more effective use of Docker is in our future work plans. By activating the use of Docker we aim to make deployments more efficient and effective.


[1] J. Humble. (2016) The case for continuous delivery. [Online]. Available:

[2] A. Morrison and B. Parker. (2016) An aerospace industry

cios move toward devops and a test-driven development envi-
ronment. [Online]. Available:

[3] T. Z. . A. Walgren. (2016) Huaweis cd transformation journey. [Online]. Available:

[4] M. Pais. (2016) Continuous delivery stories. [Online].


[5] Wikipedia. (2016) Systems development life cycle. [Online]. Available:

[6] D. F. Jez Humble, Continuous Delivery: Reliable Software Releases Through Build, Test, and Deployment Automation. Addison Wesley, 2011.

[7] C. L. Juha Itkonen, “Perceived benefits of adopting continuous deliv- ery practices,” 10th International Symposium on Empirical Software Engineering and Measurement (ESEM). Ciudad Real, Spain: ACM, 2016.

[8] K. Morris. (2016) Continuous delivery vs. traditional agile. [Online]. Available:

[9] M. Fowler. (2016) Continuousdelivery. [Online]. Available:

[10] K. Beck, Extreme programming explained : embrace change,ser. The XP Series. Boston, San Francisco, Paris: Addison-Wesley, 2000, autres tirages : 2001, 2004. [Online]. Available:

[11] B. Fitzgerald and K.-J. Stol, “Continuous software engineering: A roadmap and agenda,” Journal of Systems and Software, no. 25, pp.19–59, 8 2015.

[12] C. Tozzi. (2016) Continuous integration vs. continuous delivery: Theres an important difference. [Online]. Available:

[13] L. Chen, “Continuous delivery huge benefits, but challenges too,” IEEE Software, 3 2015.

[14] J. Allen. (2016) Patterns for continuous delivery. [Online]. Available:

[15] N. Bozic. (2016) Continuous delivery through pipelines.

[Online]. Available:

[16] (2016) Jenkins. [Online]. Available:

[17] (2016) Appium introduction. [Online]. Available:

[18] (2016) Apache jmeter. [Online]. Available:

[19] (2016) Blazemeter. [Online]. Available:

[20] (2016) Sonarqube. [Online]. Available:

[21] (2016) Jfrog. [Online]. Available:

[22] (2016) Liquibase db change management. [Online]. Available:

[23] (2016) Docker. [Online]. Available: