PHPStorm Inspections for your Continuous Integration Process

Did you know that PHPStorm (or any other Jetbrains IDE) can run inspections from command line and generate XML files for the results? This is a great “hidden” feature of those IDEs and machine-readable output means it can be somehow integrated with a continuous integration (CI) process. So let’s do this!

Inspection Profile

First thing you need is an inspection profile. I recommend creating one in your IDE clicking together the inspections and error levels as you like, so you can see the results instantly annotated in the code editor. When you’re happy with it, save the inspection profile to the project and get the inspections configuration file from .idea/inspectionProfiles. Then, commit it to your repository, doesn’t matter where you locate it, you can also keep it under it’s original path if you want to share it with all other developers – which is not a bad idea at all.

Setting Up PHPStorm on a Server

1) Download the Linux package of PHPStorm from the offical website.

2) Unpack it to some folder on your server.

3) Edit bin/idea.properties as follows:

idea.config.path=${idea.home.path}/profile/config
idea.system.path=${idea.home.path}/profile/system
idea.plugins.path=${idea.home.path}/profile/plugins
idea.log.path=${idea.home.path}/profile/log

This step is optional. It hard-links profile relative to the PHPStorm folder, effectively making it a “portable” installation. Otherwise the profile is located in the current users home directory, which is a bit problematic if you want to run PHPStorm with different users. The portable setup also allows you to easily copy that folder between servers.

4) Edit bin/phpstorm64.vmoptions to increase XmS and Xmx memory (optional).

5) Run bin/inspect.sh once, this initializes the profile folder and will fail because of a missing license.

6) Copy phpstorm.key into profiles/config folder. The key file must be created from an “Activation Code”, which can be retrieved from the JetBrains website by logging into the account and downloading the “Activation code for offline usage”. After entering the code in your desktop IDE, the file can be copied from the local profile folder to the server.

7) Run bin/inspect.sh again, the license error should be gone and the command line options are listed.

Congratulations, now you have a headless PHPStorm on your server.

Plugins

You might want to install some plugins to make some false-positive inspection errors vanish. For example, the PHP Annotations plugin is useful to make it understand use statements for annotations, instead of detecting them as “unncessary”. Plugins can be downloaded from the JetBrains website and must be unpacked into the profile/plugins folder. Plugins located in that folder are automatically enabled, no additional config necessary.

If you want to disable bundled plugins, add a disabled_plugins.txt to the config folder. Ideally, disable the plugins in the desktop IDE and copy the content to the server.

Running Inspections

Running inspections works as described on the JetBrains website. But there’s a few things you should know.

The .idea Folder Issue

You’re litterally running the IDE and therefore it does the same thing as the desktop IDE when opening a directory, it’s looking for an .idea folder

If you have no .idea folder there, no problem, it will just take all of the code that’s present in the directory.

If you have an .idea folder there – even when it’s empty – it will think “oh, that’s an IDEA-type project, I know how to read that”. It will look for the modules and directory configuration (modules.xml and *.iml file). If they’re present, no problem. If they’re missing, well, it will ignore all code because your project technically doesn’t have any modules.

I’d recommend providing it an .idea folder in any case (even when it’s not part of the repository, add it from somewhere before starting inspections), because it helps PHPStorm to understand the project better and you can tell it to ignore unimportant stuff, which makes the start-up and indexing faster. My recommendation:

  • Provide at least modules.xml and the *.iml file
  • Provide webResources.xml if you require any “Resource Root” paths
  • Provide php.xml to set the PHP language level
  • Provide misc.xml to set the JS language level

Some code examples to give you an idea how these files look like:

modules.xml

<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
    <component name="ProjectModuleManager">
        <modules>
            <module fileurl="file://$PROJECT_DIR$/.idea/my-project.iml" filepath="$PROJECT_DIR$/.idea/my-project.iml" />
        </modules>
    </component>
</project>

my-project.iml

<?xml version="1.0" encoding="UTF-8"?>
<module type="WEB_MODULE" version="4">
  <component name="NewModuleRootManager">
    <content url="file://$MODULE_DIR$">
      <sourceFolder url="file://$MODULE_DIR$/src" isTestSource="false" packagePrefix="Foo\" />
      <sourceFolder url="file://$MODULE_DIR$/test" isTestSource="true" />
      <excludeFolder url="file://$MODULE_DIR$/build" />
      <excludePattern pattern="*.csv" />
    </content>
    <orderEntry type="inheritedJdk" />
    <orderEntry type="sourceFolder" forTests="false" />
  </component>
</module>

webResources.xml

<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
  <component name="WebResourcesPaths">
    <contentEntries>
      <entry url="file://$PROJECT_DIR$">
        <entryData>
          <resourceRoots>
            <path value="file://$PROJECT_DIR$/src/js-modules" />
          </resourceRoots>
        </entryData>
      </entry>
    </contentEntries>
  </component>
</project>

php.xml

<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
    <component name="PhpProjectSharedConfiguration" php_language_level="7.2" />
</project>

misc.xml

<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
    <component name="JavaScriptSettings">
        <option name="languageLevel" value="JSX" />
    </component>
</project>

-d Option Limitations

The -d option can only be passed once (Upvote!). If you pass it multiple times, only the last one will be inspected. And you cannot target files with the -d option (Upvote!), it only takes directories.

To work around this limitation, JetBrains suggests to use scopes. You need to have a scope defined, which is done via .idea/scopes/ScopeName.xml looking as follows. The file patterns can be clicked together in the desktop IDE when managing scopes.

<component name="DependencyValidationManager">
  <scope name="ScopeName" pattern="here some file patterns" />
</component>

Unfortunately, there is no easy way to make it use a scope, so you have to do it the hard way – via JVM options. The best way is to make use of the PHPSTORM_VM_OPTIONS environment variable to let PHPStorm read additional JVM options from a file. In this example, we create a file called build/inspect.vmoptions that contains:

-Didea.analyze.scope=ScopeName

Then, we can start inspect.sh like this.

PHPSTORM_VM_OPTIONS=build/inspect.vmoptions bin/inspect.sh

All of this could be generated before starting the inspections, so you have everything you need to run inspections on arbitrary lists of files.

Stale Caches

When you’re switching branches a lot or have a lot of change in that project in general, PHPStorm might run into the issue of stale caches, no longer being able to run inspections properly. The fix is to delete the profile/system/caches and profile/system/index folders to force a re-index. Of course, running inspections will take more time then.

Single Process Limitation

The IDE allows a single process only, so you cannot run multiple inspections in parallel. Having multiple PHPStorm installations with the portable configuration (as shown above) can serve as a workaround, you can then have a process running from each installation.

The Endless Loop

Sometimes the inspection process gets stuck in an endless loop, trying to inspect the same file over and over again, never finishing. I’ve seen this happening especially with large data files like CSV or XML. Fortunately, it is not much of a problem, since inspections are executed multi-threaded and only single threads get stuck while the remaining ones are finishing the job. I worked around it by killing the process after a certain time. XML output is written on-the-fly, so you only need to fix the XML files, which are missing the closing element. Excluding these problematic files types (as seen in the my-project.iml example) also helps.

Publishing Result

If you’re running JenkinsCI, the Warnings Next Generation Plugin is the way to go. It supports parsing for IDEA inspections XML out of the box. One thing that I want to point out here, you can publish multiple reports from the same inspection results and apply certain filters. Here’s what I have in the Jenkinsfile to publish a report for PHP files and another one for JavaScript files:

recordIssues enabledForFailure: true, tool: ideaInspection(pattern: 'build/phpstorm/*.xml', id: 'idea_php', name: 'PHP Inspections'), filters: [includeFile('.*.php')]
recordIssues enabledForFailure: true, tool: ideaInspection(pattern: 'build/phpstorm/*.xml', id: 'idea_js', name: 'JS Inspections'), filters: [includeFile('.*.js'), excludeFile('.*src/js-legacy/.*')]

If you’ve used the -d option, the file paths in the resulting XML files will be prefixed with file://. Because of this, the plugin cannot resolve the file paths and therefore does not link into the source code. I guess, this is going to be fixed at some point, but until then fix it yourself with sed before publishing:

sed -i -- "s/file:\/\///g" build/phpstorm/*.xml

If you cannot use that plugin for some reason, or you want to do some more filtering, you might want to consider scheb/idea-inspections-checkstyle-converter, which is mainly converting the IDEA XML format to the more common Checkstyle format, but also comes with some more filtering options.

Summary

That’s all you need to integrate PHPStorm inspections into the CI process. Obviously, some things are not as easy as they should be, but if you know about the pitfalls (what you do, now after reading this posts), it’s realtively straight-forward. I hope this post will help you integrating them into your development process and you can gain some increased code quality from it over time.

7 thoughts on “PHPStorm Inspections for your Continuous Integration Process

  • March 4, 2019 at 07:27
    Permalink

    Is this legal?

    I don’t think the software license covers anything beyond individual or team use? Putting an IDE on your server may violate the license.

    It’s the main reason we didn’t go this route, and one reason we’re thinking of switching to Psalm for both IDE and CI inspections.

    Reply
    • March 4, 2019 at 14:20
      Permalink

      I’m not a legal expert, but if you treat the server as a “user” and you have a separate license for that instance I’d say its fine. The only thing that is a bit in a grey-zone is running multiple instances of PHPStorm with the same license on that server.

      Reply
    • March 5, 2019 at 09:58
      Permalink

      Thank you for clarifying this, so far this was quite a grey area for our community.

      Reply
    • March 5, 2019 at 14:05
      Permalink

      Please accept my apologies. I’ve confused the usecases and have supplied incorrect information. Our team will provide clarification on this subject later this month.

      Reply
  • March 5, 2019 at 15:45
    Permalink

    Hi.
    I did a similar thing a while ago. I think that all the other linter/CA tools for PHP are lagging behind with their rules.
    I made a docker container with phpStorm inside.
    I also wrote some PHP code to transform the CA result into a format that can be read by Teamcity.
    It works well but sometimes I had issues that it would report vendor classes as “undefined” errors. The same CA rule would pass locally.
    Here is my proof of concept code for Teamcity: https://github.com/batmaid/teamcityCodeAnalysis

    Reply
  • March 30, 2019 at 22:21
    Permalink

    Similar results can be achieved with “bare” php packages in 2019

    – Coding style: Easy Coding Standard or any other
    – Static analysis reports: PHPStan
    – Dead code, code quality: Rector

    Reply
    • March 31, 2019 at 01:36
      Permalink

      True, there’s definitely other tools which can achieve a similar result.

      At our company it’s quite useful, because almost everyone is using PHPStorm for PHP development and we have a inspection profile that everyone uses. So it’s great to run inspections in the CI process and get a report for the exact same errors everyone sees in their IDE.

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.