SemiAutoRecon: New Reconnaissance Tool

What is a SemiAutoRecon tool?

SemiAutoRecon is a multi-threaded network reconnaissance tool. It is used for semi-automated service enumeration. The features of this tool are all very customizable. It is meant to be used as a time-saving tool in CTFs and other penetration testing scenarios, for example, the OSCP exam). It is most emphatically useless in real-world interactions. This fantastic tool was created by Tib3rius, and it is the successor of the autorecon tool.

Why use the SemiAutoRecon tool?

SemiAutoRecon was inspired by three tools used during the OSCP labs by the author: Reconnoitre, ReconScan, and bscan. While all three tools were beneficial, none of them alone provided the needed capabilities. SemiAutoRecon combines the most satisfactory characteristics of the existing tools while also introducing numerous additional capabilities to help testers with multiple target enumeration.

Note: Use this tool with the owner’s consent of the web application or website. The author will not be held liable for any harmful actions due to the incorrect usage of this tool.

How does the SemiAutoRecon tool work?

The tool operates by first doing port scans and service detection scans. Based on those first findings, the tool will perform additional enumeration scans of those services using various technologies. For example, if HTTP is discovered, feroxbuster and several others will be launched if the user grants permission.

Who can use the SemiAutoRecon tool?

There are some types of people who can use the SemiAutoRecon tool:

• CTF competition
• Cybersecurity professionals (Penetration Tester, Red Team)
• Bug Hunters
• Students in cybersecurity examination (OSCP exam)

How does the SemiAutoRecon tool work following the OSCP exam?

To comply with OSCP exam regulations, the tool’s default setup does not execute any automated exploitation. You do it at your own risk if you want to add semi-automatic exploit tools to the settings. 

Features of SemiAutoRecon tool

• Before executing any command, prompts the user.
• Multiple targets are supported in the form of IP addresses, IP ranges (CIDR notation), and resolvable hostnames. IPv6 connectivity is also available.
• It can scan many targets simultaneously, using multiple CPUs if available.
• The advanced plugin system makes it simple to create new scans.
• Port scanning plugins that may be customized for greater flexibility in your first scans.
• Service scanning plugins that may be customized for further enumeration.
• Suggested manual follow-up commands for situations when automation makes no sense.
• Possibility of restricting port scanning to a mix of TCP/UDP ports.
• Ability to bypass the port scanning process by providing information about services that should be available.
• Pattern matching on a global and per-scan approach emphasizes and recovers important information from noise.
• A simple directory structure for storing results.
• Full logging of instructions executed, including errors if they fail.
• A powerful configuration file allows you to utilize your preferred settings every time.
• A plugin labelling system that allows you to add or remove specific plugins.
• You can set global and per-target timeouts if you only have a limited time.
• There are four degrees of verbosity, adjustable by command-line parameters and Up/Down arrows during scanning.
• Colorized output is used to distinguish between different types of information. It is possible to disable it for accessibility concerns.

How to install the SemiAutoRecon tool in Linux?

SemiAutoRecon tool does not come pre-installed in Kali Linux and Parrot Security OS. To use this tool, the user may require to install it manually. Before installing this tool, the user may install necessary dependencies and packages.

You have to update all other old repositories when installing any new tool.

sudo apt-get update

SemiAutoRecon requires Python 3.7 or above, as well as pip.

sudo apt-get install python3

sudo apt-get install python3-pip

SecLists is a collection of several sorts of lists used during security assessments gathered in one location. It contains usernames, passwords, URLs, sensitive data patterns, fuzzing payloads, web shells, and many other list kinds are available.

sudo apt-get install seclists

Users may need to install other dependencies or packages depending on the operating system. Some of the packages are pre-installed in Kali Linux and Parrot Security OS.

sudo apt-get install curl

sudo apt-get install enum4linux

sudo apt-get install feroxbuster

sudo apt-get install gobuster

sudo apt-get install impacket-scripts

sudo apt-get install nbtscan

sudo apt-get install nikto

sudo apt-get install nmap

sudo apt-get install onesixtyone

sudo apt-get install oscanner

sudo apt-get install redis-tools

sudo apt-get install smbclient

sudo apt-get install smbmap

sudo apt-get install snmpwalk

sudo apt-get install sslscan

sudo apt-get install svwar

sudo apt-get install tnscmd10g

sudo apt-get install whatweb

sudo apt-get install wkhtmltopdf

To check whether all the packages are installed or not in Kali Linux and Parrot Security OS in one single command. If any package is not installed, then it will install that specific package.

sudo apt-get install seclists curl enum4linux feroxbuster gobuster impacket-scripts nbtscan nikto nmap onesixtyone oscanner redis-tools smbclient smbmap snmp sslscan sipvicious tnscmd10g whatweb wkhtmltopdf

Different methods for installing the SemiAutoRecon tool

#1. pipx method

SemiAutoRecon should be installed using pipx, and it is a recommended method. SemiAutoRecon will be installed in its own virtual environment and made available in the global context, eliminating conflicting package dependencies and the resultant instability. 

To begin, use the following commands to install pipx:

sudo apt install python3-venv

python3 -m pip install --user pipx

python3 -m pipx ensurepath

To utilize pipx, you must re-source the shell you are using in the terminal (/.bashrc or /.zshrc file) performing these instructions. To re-source, follow the command:

source ~/.bashrc

source ~/.zshrc

SemiAutoRecon may be installed with the following command:

pipx install git+https://github.com/Tib3rius/SemiAutoRecon.git

Please keep in mind that if you wish to run this tool for quicker SYN scanning and UDP scanning, you must run it as the root user or using sudo. The following examples are:

sudo env "PATH=$PATH" semiautorecon [options]

sudo $(which semiautorecon) [options]

#2. pip method

You can also install SemiAutoRecon with pip by typing the command:

python3 -m pip install git+https://github.com/Tib3rius/SemiAutoRecon.git

It is important to note that if you wish to run this tool for quicker SYN scanning and UDP scanning, you must run it as the root user or using sudo.

SemiAutoRecon, like pipx, maybe started by simply running semiautorecon if installed through pip.

#3. Manual method

If you don’t want to use pip or pipx, you can always install and run semiautorecon.py as a script. 

Install the following dependencies from the SemiAutoRecon directory:

python3 -m pip install -r requirements.txt

After that, you’ll be able to run the semiautorecon.py script:

python3 semiautorecon.py [options] 127.0.0.1

The IP address of the localhost is 127.0.0.1 in this case.

Updating and Upgrading SemiAutoRecon tool

#1. pipx

Upgrading SemiAutoRecon once installed with pipx is the simplest and most recommended option. 

To begin, enter the following command into your terminal:

pipx upgrade semiautorecon

#2. pip

If you installed SemiAutoRecon using pip, you must first delete it and then reinstall it with the same install command:

python3 -m pip uninstall semiautorecon

python3 -m pip install git+https://github.com/Tib3rius/SemiAutoRecon.git

#3. Manually

If you manually installed SemiAutoRecon, simply navigate to the SemiAutoRecon directory and execute the following command:

git pull

Assuming you did not change anything in the SemiAutoRecon directory, this should fetch the most recent code from this GitHub repo. You may run SemiAutoRecon, as usual, using the semiautorecon.py script.

Plugins

A plugin update procedure is being developed. After updating, delete the /.config/SemiAutoRecon directory and run SemiAutoRecon with any parameter to repopulate it with the most recent contents.

If you rely on the config.toml file in /.config/SemiAutoRecon (i.e. you have made changes to it), just delete everything else in /.config/SemiAutoRecon except the config.toml file including the VERSION-x.x.x file.

Options and Usage

usage: semiautorecon [options]

#1. Positional arguments:

targets- Enter IP addresses (for example, 127.0.0.1), CIDR notation (for example, 127.0.0.1/24), or resolvable hostnames (for example, example.com) to scan.

#2. Optional arguments:

-t TARGET_FILE, –target-file TARGET_FILE- Read multiple targets from file.

-p PORTS, –ports PORTS- Input comma-separated list of ports/port ranges to scan. TCP/UDP ports can be specified by prepending the list with T:/U:. To scan both TCP and UDP, enter port(s) at the beginning or specify B: For example, 53,T:21-25,80,U:123,B:123. By default, the value is None.

-m MAX_SCANS, –max-scans MAX_SCANS- The maximum number of concurrent scans to run. By default, the value is 50.

-mp MAX_PORT_SCANS, –max-port-scans MAX_PORT_SCANS- The maximum number of concurrent port scans to run. By default value is 10 (approx 20% of max-scans unless specified)

-c CONFIG_FILE, –config CONFIG_FILE-The location of SemiAutoRecon’s config file. By default the path is ~/.config/SemiAutoRecon/config.toml

-g GLOBAL_FILE, –global-file GLOBAL_FILE- The location of SemiAutoRecon’s global file. By default the path is ~/.config/SemiAutoRecon/global.toml

–tags TAGS- The tags are used to determine which plugins should be included. To combine tags, use a plus symbol (+) to separate them. To make numerous groups, separate them with a comma (,). A plugin must have all of the tags listed in at least one group to be included.

–exclude-tags TAGS- Tags specify which plugins should be excluded. To join tags together, use a plus symbol (+) to separate them. To make numerous groups, separate them with a comma (,). A plugin must have all of the tags given in at least one category to be excluded.

–port-scans PLUGINS- Override —tags / —exclude-tags for the PortScan plugins specified (comma separated).

–service-scans PLUGINS- Override —tags / —exclude-tags for the ServiceScan plugins specified (comma separated).

–reports PLUGINS- Override –tags / –exclude-tags for the listed Report plugins (comma separated).

–plugins-dir PLUGINS_DIR- Enter the location of the plugins directory. By default, the path of the directory of the plugin is ~/.config/SemiAutoRecon/plugins

–add-plugins-dir PLUGINS_DIR- Enter the location of an additional plugins directory to add to the main one.

-l [TYPE], –list [TYPE]- List all plugins or all plugins of a particular kind. For example, –list, –list port, and –list service.

-o OUTPUT, –output OUTPUT- Enter the path where the user wants to save the output in the directory for results. By default, it will keep in the results directory.

–single-target- Scan just one target at a time. There will be no directory named after the target. The directory structure will instead be established within the output directory. By default, the value is False.

–only-scans-dir- For results, only create the “scans” directory. Other folders (for example, exploit, loot, and report) will not be made. By default, the value is False.

–no-port-dirs- Don’t make port directories (e.g., scans/tcp80, scans/udp53). Instead, keep all results in the “scans” directory. By default, the value is False.

–heartbeat HEARTBEAT- The heartbeat interval (in seconds) for scan status messages is specified with this option. By default, the value is 60.

–timeout TIMEOUT- It specifies the maximum amount of time in minutes that SemiAutoRecon should run for.

–target-timeout TARGET_TIMEOUT- It specifies the maximum time in minutes that a target should be scanned for before abandoning it and moving on.

–nmap NMAP- In scans, override the {nmap_extra} variable. By default, -vv –reason -Pn -T4 is set.

–nmap-append NMAP_APPEND- In scans, append the default {nmap_extra} variable.

–proxychains- If you’re using proxychains to run SemiAutoRecon, this is what you’ll need. By default, the value is False.

–disable-sanity-checks- Disable any sanity tests that would prevent the scans from executing. By default, the value is False.

–force-services SERVICE [SERVICE …]- A list of services, separated by spaces, in the following format: http://tcp/80/https://tcp/443/https/secure

–accessible- Attempts to improve the accessibility of SemiAutoRecon output for screenreaders. By default, the value is False.

-v, –verbose- Turn on verbose output. Repeat for greater verbosity. There are four levels in the SemiAutoRecon tool.

–version- Exits after printing the SemiAutoRecon version.

  -h, –help- Display this assistance message and then depart.

#3. Plugin arguments:

These are optional arguments for specific plugins.

–curl.path VALUE- The location to curl on the webserver. By default, the path of the curl is slash (/).

–dirbuster.tool {feroxbuster,gobuster,dirsearch,ffuf,dirb}- This option is used to integrate the directory breaking tool. By default, it uses the feroxbuster tool.

–dirbuster.wordlist VALUE [VALUE …]- When directory busting, the wordlist(s) to utilize. Spaces should be used to separate multiple wordlists. By default, the wordlist’s location is ‘~/.config/SemiAutoRecon/wordlists/dirbuster.txt’.

–dirbuster.threads VALUE- When directory busting, the number of threads to execute. By default, the number of threads used is 10.

–dirbuster.ext VALUE- The extensions you want to fuzz and no dot, separated by commas. By default, the extensions are txt, html, php, asp, aspx, jsp.

–onesixtyone.community-strings VALUE- The file gives a list of possible community strings. By default:, the location is /usr/share/seclists/Discovery/SNMP/common-snmp-community-strings-onesixtyone.txt

#4. Global plugin arguments:

  These are optional arguments that can be used by all plugins.

–global.username-wordlist VALUE- A username wordlist that may be used for brute-forcing. By default, the location is /usr/share/seclists/Usernames/top-usernames-shortlist.txt

–global.password-wordlist VALUE- A password wordlist that may be used for brute-forcing. By default, the location is /usr/share/seclists/Passwords/darkweb2017-top100.txt

–global.domain VALUE- The domain name to utilize (if known). DNS and/or Active Directory are both supported.

Levels of verbosity

SemiAutoRecon has four verbosity levels:

• (none) Minimal output- SemiAutoRecon will notify you when scanning targets begin and end.
• (-v) Verbose output- SemiAutoRecon will notify you when plugins begin to execute and report open ports and detected services.
• (-vv) Very verbose output- SemiAutoRecon will also specify the specific commands that plugins are running, highlight any matched patterns in the command output, and notify when plugins terminate.
• (-vvv) Very, very verbose output- Everything will be produced by SemiAutoRecon. Every line of every command that is presently executing. When scanning numerous targets simultaneously, this might result in excessive output. It is not recommended to use -vvv unless you need to observe live command output.

Note: You can change the verbosity of SemiAutoRecon mid-scan by pressing the up and down arrow keys.

How the results are stored?

The results will be saved in the./results directory by default. For each target, a new subdirectory is generated. This subdirectory’s structure is as follows:

├── exploit/
├── loot/
├── report/
│ ├── local.txt
│ ├── notes.txt
│ ├── proof.txt
│ └── screenshots/
└── scans/
├── _commands.log
├── _manual_commands.txt
├── tcp80/
├── udp53/
└── xml/

Practical Video