Last updated at Tue, 25 Jul 2017 18:10:12 GMT
Over the last few weeks the Metasploit team at Rapid7 has engaged in an overhaul of our development process. Our primary goals were to accelerate community collaboration and better define the scopes of our open source projects. The first step was to migrate all open source development to GitHub. This has resulted in a flood of contributors and lots of great new features and content.
One controversial change involved removing old, buggy automation tools that simply didn't meet the quality bar, or our scope for the framework. This resulted in the removal of file_autopwn and db_autopwn. Both of these modules were easy to use, but were more likely to fall over and crash than produce useful results. The db_autopwn code started off as a joke and never reached a point where it was actually stable. For anyone who really wants to use db_autopwn, a community contributor maintains it as a plugin in a GitHub fork.
The Metasploit products (inlcuding the open source Metasploit Framework) support automation at multiple levels. How you automate the product depends on what type of task you are working on and the granularity needed. The list below is not comprehensive; there are an infinite number of ways to extend, include, and automate Metaspoit, but these are the best supported and most common methods.
The Metasploit Console
The console (msfconsole or msfpro) supports basic automation using Resource Scripts. These scripts contain a set of console commands that are executed when the script loads. In addition to basic console commands, these scripts are also treated as ERB templates. ERB is a way to embed Ruby code directly into a document. This allows you to call APIs that are not exposed via console commands and even programmatically generate and return a list of commands based on your own logic. Resource Scripts can be specified with the -r option to the Metasploit Console and ~/.msf4/msfconsole.rc is automatically executed on startup if it exists. Resource Scripts can also be executed from the console prompt through the resource command. For more on this approach, see Automating the Metasploit Console.
The console (msfconsole or msfpro) also supports the concept of Plugins. Plugins add new console commands that provide a utlity or automation function. The flexibility of the Ruby language allows Plugins to do nearly anything, from exposing new automation capabilities, to providing socket-level content filtering to prevent the tripping of a remote IDS. Direct integration with Nexpose, Nessus, and OpenVAS from the console are accomplished through plugins. The full list of default plugins can be found in the GitHub repository. Plugins are the suggested way to work on new console commands and share them with the wider community.
Auxiliary Module Custom Commands
Auxiliary modules are defined as any Metasploit module that performs a remote operation of some sort, but doesn't take an actual payload like an exploit. Auxiliary modules handle things like reconnaisance, authentication bypass, network sniffing, and vulnerability discovery. One little-used feature of Auxiliary modules is the ability to define new console commands from within the module context. The user would enter "use auxiliary/module/name" and if the module exposes new commands, these would become available to the console. One example is the TrendMicro ServerProtect File Access module.
Custom Auxiliary Modules
Although we do not accept modules that run other modules into the Metaspoit Framework proper, these are trivial to create as custom modules and allow for any form of automation, exposed through any supported user interface. The major advantage to writing automation tools as Auxiliary modules is that they will usually work just fine from Metasploit Community Edition or Metasploit Pro, as well third-party interfaces like MSFGUI. One example of an existing automation module in the framework (and one that is still being reviewed from a design perspective) is browser_autopwn. This module will automatically configure exploit modules and redirect the target to the appropriate one. The reason why this isn't really a good fit for the framework is that payload and target selection are hardcoded to values that may not always work. We are looking into better ways to handle client-side exploit automation, but until then, it serves as an in-tree example of Auxiliary module automation.
Metasploit Remote API
The Metasploit Framework and Metasploit Pro both support automation using a documented Remote API. On the framework side, this exposes a wide range of functionality at the lowest level, allowing the caller to run modules, interact with sessions, and generally access the backend of the Metasploit instance. Metasploit Pro builds on this by offering access to the commercial product features through the same API. In fact, the Metasploit Pro user interface uses this same API to drive the backend automation. The product was built with automation and extensibility in-mind. Using Metasploit Pro with the Remote API makes it painless to remotely automate a penetration test, across multiple instances of Pro, all from a central location. Rapid7 customers are using this today to conduct automated exploitation during off-hour scan windows and to automate things like password testing across dozens of remote sites at once, with centralized reporting. You can find examples of the Pro API automation in the documentation directory of the framework. The msfrpc-client GEM is available for Ruby developers.
At the end of the day, the Metasploit Framework is a development environment more than it is a standalone product. The APIs offered make it easy to embed a copy of the framework into another tool, parse the module database looking for a specific set of criteria, or even repurpose the existing network APIs to build something new. Each of the previous methods makes it easy to load custom Ruby code and leverage that code in a useful way; to get the most out of the Metasploit products, it helps to become familiar with the framework API itself. Nearly all of the framework code is available under an open source license and the latest changes can be found in the main GitHub repository. Metasploit Pro customers are encouraged to contact support (and likely, from there, the development team) about any ideas they have for development or integration. Nearly any code written for the Metasploit Framework is drop-in compatible with Metasploit Pro.
If you have any questions about automation, the Discussion forum in the Rapid7 Community is a great way to get started. For realtime discussion, the #metasploit channel on the FreeNode IRC network (#metasploit) is a great resource as well.