Overview of OpenRewrite and Moderne

Diving into the world of code modernization can often feel like navigating a maze without a map, especially when it comes to refactoring and ensuring that your codebase doesn't just work, but works well. 

Enter OpenRewrite—a tool that lets developers automate tedious code changes, framework migrations, and security upgrades. OpenRewrite was created to make effortless code management a reality for developers.

But why stop there? Scaling up the OpenRewrite concept to handle the vast expanses of enterprise-level codebases, we developed Moderne. The Moderne Platform is not just about upgrading a repository here and there; it's about taking on thousands at a time, giving developers the power they need to transform and secure their code faster than ever before. 

With OpenRewrite and Moderne, our goal has been to revolutionize the daunting task of code refactoring. Let's explore how OpenRewrite and Moderne each turn the chaos of code management into straight-forward, automated tasks.

OpenRewrite overview

OpenRewrite is an open-source project that offers semantic analysis and refactoring of code, so that everyone can use and contribute to it, over time composing more and more refactoring operations that make whole framework migrations possible. It enables individual developers to search and fix code in one repository at a time. OpenRewrite is integrated with build tooling and can be plugged into different workflows, from CI integrations to mass refactoring of multiple microservices/repositories. 

OpenRewrite originated at Netflix, where Jonathan Schneider, creator and founder of OpenRewrite, worked in engineering tools. Netflix’s culture of freedom and responsibility led to code that didn’t conform to any single style. It also prevented a central team from imposing any gates on product engineers, so they couldn’t just break the build when they saw a pattern they didn’t like. 

When a central team asked product teams to develop in a specific way, what they heard repeatedly was, “I do not have time to deal with this, but if you do it for me, I’ll merge.” So eventually, Jonathan took it literally and built automation to affect such changes. Prior to OpenRewrite, Jonathan created the Gradle Lint plugin, a popular plugin that manipulates Groovy build files and upgrades dependencies. Since it manipulated Groovy code backing build configuration, why not also source code?

The usage of OpenRewrite expanded to Java when the platform team wanted to stop maintaining an internal logging library, which originated before SLF4J, and replace it with standard SLF4J. At the time, it had been deprecated for six years, but because of Netflix’s culture, there were still countless references to this library in the Netflix codebase. With the adoption of OpenRewrite automation, it was removable at scale.

In the years since, OpenRewrite’s capabilities have been expanded from simple API changes to more complex framework migrations and CVE patching.

How OpenRewrite operates/internals

OpenRewrite manipulates a Lossless Semantic Tree (LST) representation of code. LSTs are a foundational piece of what makes OpenRewrite so powerful. An LST goes beyond abstract syntax trees (ASTs). It ensures refactoring is accurate, with no false positives. The LST has special characteristics that allow it to parse code and generate code back into the standard text representation for performing accurate transformations and searches.

OpenRewrite LSTs are produced by guiding the compiler through the first two phases of compilation to generate a compiler type-attributed LST, but then maps it to the OpenRewrite LST that also preserves formatting. Here are the unique characteristics of an OpenRewrite LST: 

• Type-aware – allowing 100% correct semantic code analysis and transformation.
• Style-preserving – so transformations produced are idiomatic within the projects to which they are applied. Therefore, the same transformation applied to multiple projects will look potentially different. 

OpenRewrite calls a single code search or transformation operation a recipe. OpenRewrite provides a lot of building block recipes like “find method,” “change method,” “find transitive dependency,” “upgrade dependency,” or “exclude dependency.” These recipes, in turn, can be composed into more complex recipes by grouping them together into a composite recipe. 

When the building blocks are not enough, a recipe can be written as a program in the same language as the code we want to transform, allowing us to encapsulate complex logic with the full expressiveness of the language already familiar to developers. We don’t need to learn a new domain-specific language (DSL) or programming language.

These building blocks abstract away many of the details to ensure that edits that we make to source code obey the original style of the project. For example, our composite JUnit 4 to 5 recipe contains the following transformations:

type: specs.openrewrite.org/v1beta/recipe
name: org.openrewrite.java.testing.junit5.JUnit4to5Migration
displayName: JUnit Jupiter migration from JUnit 4.x
description: Migrates JUnit 4.x tests to JUnit Jupiter.
recipeList:
    - org.openrewrite.java.ReorderMethodArguments:
        methodPattern: org.junit.Assume assume*(String, boolean)
        newParameterNames: [b,message]
        oldParameterNames: [message,b]
    - org.openrewrite.java.ChangeType:
        oldFullyQualifiedTypeName: org.junit.Assume
        newFullyQualifiedTypeName: org.junit.jupiter.api.Assumptions
    - org.openrewrite.java.testing.junit5.RemoveObsoleteRunners:
        obsoleteRunners: [org.junit.runners.JUnit4,org.junit.runners.BlockJUnit4ClassRunner]
    - org.openrewrite.maven.RemoveDependency:
        groupId: junit
        artifactId: junit
        scope: null
    - org.openrewrite.java.testing.junit5.IgnoreToDisabled
    - org.openrewrite.java.testing.junit5.UseHamcrestAssertThat
    - org.openrewrite.java.testing.junit5.UseMockitoExtension
    - org.openrewrite.java.testing.junit5.UseTestMethodOrder
    - org.openrewrite.java.testing.junit5.MigrateJUnitTestCase
    - org.openrewrite.java.testing.junit5.AssertToAssertions
    - org.openrewrite.java.testing.junit5.CategoryToTag
    - org.openrewrite.java.testing.junit5.CleanupJUnitImports
    - org.openrewrite.java.testing.junit5.ExpectedExceptionToAssertThrows
    - org.openrewrite.java.testing.junit5.TemporaryFolderToTempDir
    - org.openrewrite.java.testing.junit5.TestRuleToTestInfo
    - org.openrewrite.java.testing.junit5.UpdateBeforeAfterAnnotations
    - org.openrewrite.java.testing.junit5.UpdateTestAnnotation
    - org.openrewrite.java.testing.junit5.ParameterizedRunnerToParameterized
    - org.openrewrite.java.testing.junit5.JUnitParamsRunnerToParameterized
    - org.openrewrite.java.testing.junit5.MockitoJUnitToMockitoExtension
    - org.openrewrite.java.testing.junit5.ExpectedExceptionToAssertThrows
    - org.openrewrite.java.testing.junit5.UpdateMockWebServer
    - org.openrewrite.java.testing.hamcrest.AddHamcrestIfUsed

And more. You can view the whole recipe here https://github.com/openrewrite/rewrite-testing-frameworks/blob/main/src/main/resources/META-INF/rewrite/junit5.ym

One curious thing about OpenRewrite recipes is that it started by affecting code transformations and backed into search from there to address the common request of, “I'd like to first search for this problem in my code to see how pervasive it is before I write a refactoring recipe for it.” So it treats search like a special case of transformation by just adding markers on LST elements that a recipe finds. Then, it can be hydrated into text however the developer wishes (usually as comments that can contain TODOs, Jira issue numbers, etc.).

To identify every method invocation in a whole package of Google’s Guava library, we can instruct OpenRewrite to perform this search:

The results (see below) show it isn’t looking for methods with particular names. OpenRewrite can actually prove that these method calls come from the desired package: 

This recipe in plain OpenRewrite YAML looks like:

type: specs.openrewrite.org/v1beta/recipe
name: org.openrewrite.FindGuavaCollect
displayName: Find all uses of Guava collect package
description: Every type and every method regardless of number of arguments.
recipeList:
    - org.openrewrite.java.search.FindMethods:
        methodPattern: com.google.common.collect.* *(..)
        matchOverrides: true

Give this a try by following the OpenRewrite Quickstart Guide or one of the tutorials, such as "Automatically fix Checkstyle violations" or “Migrate to Java 21.”

The Moderne Platform overview

The Moderne Platform is a collaboration platform for developers. It allows them to understand and evolve large codebases.

Moderne provides OpenRewrite auto-refactoring and auto-remediation capabilities at an enterprise scale. It also provides automated code insights and offers many more features that enable you to secure and maintain source code with speed and accuracy across your entire codebase. It scales horizontally across many repos at once—thousands of repos with millions of lines of code—as opposed to one repo at a time with OpenRewrite.

The Moderne Platform is not built directly on top of OpenRewrite. Instead, it relies on proprietary technology to efficiently apply OpenRewrite recipes to codebases at scale, enabling you to do in minutes what might take you hours with OpenRewrite. 

OpenRewrite is computationally expensive: it takes source code as text, produces LSTs for it using the compiler, applies the recipe, and produces new source code as text. That means that for each recipe run, the OpenRewrite build tools must start from scratch.

The Moderne Platform batch-builds and serializes LSTs, and stores them on disk, enabling fast, real-time actions and analysis across your codebase all the time. This means that LSTs are built once, and then shared across teams for whatever work they are doing. It results in significant time and infrastructure cost savings. 

Central teams within companies, such as platform teams, are typically responsible for managing the Moderne Platform. They configure Moderne ingest for their organization’s codebase to produce LSTs en masse. 

By operating on already produced and cached LSTs, the Moderne Platform produces OpenRewrite recipe results within minutes. Developers can search and transform code, examine diffs, and issue pull requests that can later be reviewed and put through the usual CI/CD pipeline. 

When applying open-source recipes to modernize individual repositories, the benefit of running OpenRewrite recipes far outweighs the cost. However, running recipes at scale, especially those driving large-scale migrations, means you have to make sure the recipes will work as expected.

At Moderne, we use the platform when developing and testing such recipes with the open-source community. Moreover, we run and compile them daily to make sure there are no regressions. When you don’t rely on the open-source community to supply you with recipes, and want to build your own custom migrations, the Moderne Platform is essential. Moderne tools are free for open-source maintainers, and Moderne runs a free tenant for open-source software at app.moderne.io. 

Here are some of the key benefits and features of the Moderne Platform.

Perform code impact analysis in a fraction of the time and cost of today’s methods

Traditionally, enterprise code impact analyses are expensive, lengthy, labor-intensive endeavors—costing in the millions of dollars, taking weeks or months of multiple engineers’ time to perform multiple difficult, tedious work digging into code. It takes so long that results are typically outdated before the analysis is finished since code is always moving forward. The typical reward? Impact analysis paralysis and a failed code migration or modernization effort. 

The Moderne Platform provides a new paradigm for performing large-scale code impact analyses. As a standard, built-in capability of Moderne, impact analyses can now be done in minutes—across any number of repositories and millions or billions of lines of code. The result? Impact analyses that are continuously up-to-date and actionable in real time because the capabilities are always-on. 

Two key features of the Moderne Platform aid developers, platform engineers, and data engineers in performing code impact analysis easily and inexpensively.

• Data tables – Moderne data tables (see below) are a cutting-edge feature of the Moderne platform that enable you to quickly understand the impact of code changes. They can be created in minutes and collect real-time insightful data about dependencies, CVEs, licenses, version numbers, API usage—and pretty much any other information you want to know about your source code. You can learn more about data tables in our documentation, our blog, and our demo video. 

• Code visualizations – Moderne code visualizations give real-time insights into relationships across your codebase. They enable developers to reduce complex code data into a recognizable form that can convey code impacts to business leadership. Through just a few clicks, Moderne code visualizations (see below) provide clarity and insight into desired code modernization and change management outcomes. This helps to prioritize and advocate for follow-on change. Using visualizations, developers can estimate timelines for modernization efforts and migrations more accurately. Find out more about code visualizations in our documentation, our blog, and our demo video.

Work locally on many-repos and perform auto-remediations with Moderne CLI 

The Moderne CLI (command line interface) enables developers to work locally to examine and auto-remediate code on multiple repositories. Essentially, the Moderne CLI (see below) was built to amplify the refactoring automation capacity and control that developers can have at their fingertips. The number of repos in play is only limited by the computing power of a developer’s own machine.

By using the Moderne CLI in combination with the Moderne Platform, you can accelerate developer work across your organization plus align with multiple teams and business units to operate on the codebase at unprecedented scale.

The Moderne CLI is also used to batch-build LSTs for mass ingestion into the platform. In addition, it works with the Moderne IDE integration to accelerate authoring and testing of OpenRewrite recipes.

Learn more about the Moderne CLI in our documentation and blog.

Code collaboration across your entire codebase with Moderne DevCenter

Two key features of the Moderne Platform allow developers, engineering teams, and business leaders to always know the state of their codebase and collaborate to move their codebase forward. 

• Moderne DevCenter dashboard – The Moderne DevCenter dashboard (see below) allows engineering, security, business, and C-level executives to track and understand the state of affairs of code modernization and migration efforts across every level of your enterprise. It can be set to measure coding progress from the engineering-team level for multiple teams within your organization to the business-unit level and to the corporate-wide level. 
  
  
  Areas of the dashboard can be customized to provide business impact and code health metrics of the projects your leadership teams care most about. You can choose the relevant metrics, parameters and visualizations to show for each level of your organization. But just as importantly, the dashboard is the driver of automated upgrades and fixes with just a push of a button.

• Activity View – The Activity View (see below) in the Moderne Platform allows developers, both individuals and teams, across company-wide engineering organizations to easily track recipe, commit, and visualization activities that were recently completed. This creates a more collaborative experience on the platform, saves developers time when working within the platform, and provides better visibility to engineering teams of coding activities happening across the organization. To find out more about using the Moderne Activity View, check out our documentation and demo video. (See Figure X.)

With the Moderne Platform, organizations with large code bases are now able to be hyper-efficient with code change-campaign management. You can channel your development teams’ energy into creativity and driving innovation for your business and customers.

With the Moderne Platform, developers can search and transform code in real-time, ridding code of bad patterns, dependencies, and security vulnerabilities at speed and making sure they never come back. This provides greater cost efficiency and better use of developer resources. 

The Moderne Platform is offered in three editions:

• Community Edition - Try out Moderne and work on open-source code using the Moderne Platform and CLI for free.
• Moderne Platform Editions - Work on an organization’s codebase with horizontal scalability. Use the secure Moderne SaaS Platform along with the Moderne CLI for mass code refactoring and analytics. Standard and Enterprise editions are available.
• Moderne DX Edition - Work on an organization’s codebase in a fully air-gapped environment. Includes an on-premises Central Service that integrates with local Moderne CLI instances.

To get started with Moderne, you can try the product on open-source code or book a demo with us.