Scanner Plugin

Basics

mastf.MASTF.scanners.plugin.Plugin(clazz)[source]

Register a scanner plugin.

This decorator is used to register a scanner plugin class. The class must have a name attribute, which represents the name of the scanner. The registered scanner instances can be accessed through the __scanners__ dictionary.

Usage Example:

@Plugin
class MyScanner(ScannerPlugin):
    name = "My Scanner"
    ...

# Accessing the registered scanners
for name, scanner in ScannerPlugin.all():
    print(name, scanner)

param clazz:: The scanner plugin class to register.
raises ValueError:: If the scanner’s name is null.
raises KeyError:: If the scanner is already registered.
return:: The input scanner plugin class.

class mastf.MASTF.scanners.plugin.Extension(value, names=None, *values, module=None, qualname=None, type=None, start=1, boundary=None)[source]

Enumeration class representing extensions.

This class defines different extensions as string values using the StringEnum base class. Each extension represents a specific category or functionality.

Available Extensions:

DETAILS: Represents details.
PERMISSIONS: Represents permissions.
HOSTS: Represents hosts.
VULNERABILITIES: Represents vulnerabilities.
FINDINGS: Represents findings.
COMPONENTS: Represents components.
EXPLORER: Represents explorer.

Example:

To use an extension, simply access its value using dot notation, e.g. Extension.DETAILS.

# Accessing the extension values
print(Extension.DETAILS)  # Output: "details"
print(Extension.PERMISSIONS)  # Output: "permissions"
print(Extension.HOSTS)  # Output: "hosts"
# ...and so on

Note

The values of the extensions should not be modified as they are used internally by the system.

class mastf.MASTF.scanners.plugin.ScannerPlugin[source]

context(extension: str, scan: Scan, file: File) → dict[source]

Generates the rendering context for the given extension

Parameters:: extension (str) – the extension to render
Returns:: the final context
Return type:: dict

extensions: list = []: The list of extensions this scanner supports

help = None: The help that will be displayed on the WebUI

name = None: The name (slug) of this scanner type (should contain no whitespace characters)

task = None: The task to perform asynchronously

title = None: Actual name (more details than name)

class mastf.MASTF.scanners.plugin.ScannerPluginTask[source]

Base class for scanner plugin tasks.

This class defines the common behavior and attributes of scanner plugin tasks. Subclasses should inherit from this class and override the appropriate methods to implement specific scanning functionality.

get_item(key) → object[source]

Get an item from the metadata.

This method retrieves an item from the metadata dictionary based on its type.

Parameters:: key (type | str) – The type of the item to retrieve.
Returns:: The value associated with the type, or None if not found.
Return type:: object

prepare_scan() → None[source]

Prepare the scan.

This method is called before running the scan and can be overridden in subclasses to perform any necessary preparation steps.

run_scan() → None[source]

Run the scan.

This method runs the scan by iterating over the methods starting with ‘do’ and executing them as sub-tasks. Each sub-task is logged, and any exceptions that occur are caught and logged as well.

Default Mixins

class mastf.MASTF.scanners.mixins.DetailsMixin[source]

Add-on to generate app details

If you use this mixin and you enable chart-rendering, they will be displayed on the front page of scan results.

charts: bool = True: Defines whether summary charts should be displayed on the details page.

ctx_details(scan: Scan, file: File, scanner: Scanner) → dict[source]

Returns the details context for the desired extension.

Parameters:: scan (Scan) – the scan to view
Returns:: all relevant context information
Return type:: dict

class mastf.MASTF.scanners.mixins.PermissionsMixin[source]

Add-on to generate permission lists according to the selected file

The returned data will be a list of PermissionFinding instances that store information where the permission has been found and the actual AppPermission reference.

ctx_permissions(scan: Scan, file: File, scanner: Scanner) → list[source]: Returns all permissions mapped to a specific file.

class mastf.MASTF.scanners.mixins.VulnerabilitiesMixin[source]

Add-on to generate vulnerabilites according to the selected file.

ctx_vulnerabilities(scan: Scan, file: File, scanner: Scanner) → list[source]

Returns all vulnerabilities that have been identified in the scan target.

Parameters:

project (Project) – the project instance
file (File) – the scan target

Returns:

a list of vulnerabilities

Return type:

list

class mastf.MASTF.scanners.mixins.FindingsMixins[source]

Add-on to generate a finding list according to the selected file.

ctx_findings(scan: Scan, file: File, scanner: Scanner) → list[source]

Returns all findings that have been identified in the scan target.

Parameters:

project (Project) – the project instance
file (File) – the scan target

Returns:

a list of vulnerabilities

Return type:

list

class mastf.MASTF.scanners.mixins.HostsMixin[source]

Mixin class for working with hosts in a scan.

This mixin provides methods for retrieving and manipulating hosts within a scan.

Usage:

Use ctx_hosts() to get all hosts identified within the scan target.
Use res_hosts() to get a serialized representation of hosts within the scan.

Example:

mixin = HostsMixin()
ctx_hosts_data = mixin.ctx_hosts(scan, file, scanner)
res_hosts_data = mixin.res_hosts(scan, scanner)

ctx_hosts(scan: Scan, file: File, scanner: Scanner) → list[source]

Get all hosts identified within the scan target.

Parameters:

scan (Scan) – The scan instance.
file (File) – The scan target.
scanner (Scanner) – The scanner instance.

Returns:

A list of hosts.

Return type:

list

res_hosts(scan: Scan, scanner: Scanner) → list[source]

Get a serialized representation of hosts within the scan.

Parameters:

scan (Scan) – The scan instance.
scanner (Scanner) – The scanner instance.

Returns:

A list of serialized hosts.

Return type:

list

class mastf.MASTF.scanners.mixins.ComponentsMixin[source]

Mixin class for working with components in a scan.

This mixin provides methods for retrieving and manipulating components within a scan.

Usage:

Use ctx_components() to get components statistics and elements for a scan.
Use res_components() to get a serialized representation of components within the scan.

Example:

mixin = ComponentsMixin()
ctx_components_data = mixin.ctx_components(scan, file, scanner)
res_components_data = mixin.res_components(scan, scanner)

ctx_components(scan: Scan, file: File, scanner: Scanner)[source]

Get components statistics and elements for a scan.

Parameters:

scan (Scan) – The scan instance.
file (File) – The scan target.
scanner (Scanner) – The scanner instance.

Returns:

A namespace object containing component statistics and elements.

res_hosts(scan: Scan, scanner: Scanner) → list[source]

Get a serialized representation of components within the scan.

Parameters:

scan (Scan) – The scan instance.
scanner (Scanner) – The scanner instance.

Returns:

A list of serialized components.

Return type:

list

Android Plugin

class mastf.MASTF.scanners.android_sast.AndroidTask[source]

Scanner task for performing Android-related scans.

This task is specifically designed for scanning Android applications and performs various scans such as YARA scan, manifest scan, app info scan, and code scan.

Example:

android_task = AndroidTask()
android_task(scan_task, observer)

do_app_info_scan() → None[source]

Perform a manifest scan on the Android application.

This method retrieves the manifest information of the Android application.

do_code_scan() → None[source]

Perform a code scan on the Android application.

This method creates a new scan task and initiates an asynchronous code scan.

do_manifest_scan() → None[source]

Perform a manifest scan on the Android application.

This method retrieves the manifest information of the Android application.

do_package_scan() → None[source]: Perform a heuristic scan on potential used third-party libraries.

do_semgrep_scan() → None[source]: Execute the semgrep OSS-Engine in a separate celery worker.

do_yara_scan() → None[source]

Perform YARA scan on the Android application.

This method executes the YARA code analysis on the scan task’s package directory.

prepare_scan() → None[source]

Prepare the scan by setting up the APK object.

This method initializes the APK object by loading the APK file associated with the scan.

class mastf.MASTF.scanners.android_sast.AndroidScannerPlugin[source]

Android SAST Engine Plugin.

This scanner plugin provides basic security checks for Android apps. It utilizes various mixins for different functionalities such as details, permissions, hosts, findings, and components.

Example:

android_plugin = AndroidScannerPlugin()
android_plugin.run_scan()

Note

Make sure to properly configure the scanner plugin before running the scan.

extensions: list = [Extension.DETAILS, Extension.PERMISSIONS, Extension.HOSTS, Extension.FINDINGS, Extension.EXPLORER, Extension.COMPONENTS]: The list of extensions this scanner supports

help = 'Basic security checks for Android apps.': The help that will be displayed on the WebUI

name = 'Android Plugin': The name (slug) of this scanner type (should contain no whitespace characters)

task: alias of AndroidTask

title = 'Android SAST Engine': Actual name (more details than name)

Android Scan Tasks

mastf.MASTF.scanners.android_sast.app_info_scan.get_app_info(inspector: ScannerPluginTask) → None[source]

Retrieves and saves information about the scanned app.

Parameters:: inspector – The ScannerPluginTask object.

mastf.MASTF.scanners.android_sast.app_info_scan.get_app_net_info(inspector: ScannerPluginTask) → None[source]

Retrieve network security information from XML files.

This function analyzes network_security_config.xml files present in the given directory and its subdirectories, extracting network security information from them. The extracted information is processed using a visitor pattern implemented by the AXmlVisitor class and a NetworkSecurityHandler.

Parameters:: inspector (ScannerPluginTask) – An instance of ScannerPluginTask, representing the scanner task.

class mastf.MASTF.scanners.android_sast.app_info_scan.NetworkSecurityHandler(inspector: ScannerPluginTask)[source]

mastf.MASTF.scanners.android_sast.manifest_scan.get_manifest_info(inspector: ScannerPluginTask) → None[source]

Get manifest information from the Android app.

This function collects detailed information about permissions, components, and intent filters from the AndroidManifest.xml file of the app.

Parameters:: inspector (ScannerPluginTask) – The scanner plugin task inspector.

mastf.MASTF.scanners.android_sast.manifest_scan.run_manifest_scan(inspector: ScannerPluginTask, manifest_file: Path)[source]

Run manifest scan on the AndroidManifest.xml file.

This function parses the AndroidManifest.xml file and performs a scan using the specified handler and visitor.

Parameters:

inspector (ScannerPluginTask) – The scanner plugin task inspector.
manifest_file (pathlib.Path) – Path to the AndroidManifest.xml file.

class mastf.MASTF.scanners.android_sast.manifest_scan.AndroidManifestHandler(inspector: ScannerPluginTask, path: Path)[source]

Inspects AndroidManifest files.

Parameters:

inspector – The ScannerPluginTask object for scanning.
path – The path to the AndroidManifest.xml file.

handle_component(element: Element, ctype: str, name: str) → None[source]

Handles a component element in the AndroidManifest.xml.

Parameters:

element – The component element.
ctype – The component type (e.g., service, provider, receiver).
name – The name of the component.

link(visitor: AndroidManifestVisitor) → None[source]

Links the AndroidManifestHandler with an AXmlVisitor.

Parameters:: visitor – The AXmlVisitor to link with.

on_activity(element: Element, name: str) → None[source]

Event handler for activity elements in the AndroidManifest.xml.

Parameters:

element – The activity element.
name – The name of the activity.

on_application(element: Element, name: str) → None[source]

Event handler for application elements in the AndroidManifest.xml.

Parameters:

element – The application element.
name – The name of the application.

on_permission(element: Element, identifier: str) → AppPermission[source]

Event handler for permission elements in the AndroidManifest.xml.

Parameters:

element – The permission element.
identifier – The identifier of the permission.

on_provider(element: Element, name: str) → None[source]

Event handler for provider elements in the AndroidManifest.xml.

Parameters:

element – The provider element.
name – The name of the provider.

on_receiver(element: Element, name: str) → None[source]

Event handler for receiver elements in the AndroidManifest.xml.

Parameters:

element – The receiver element.
name – The name of the receiver.

on_service(element: Element, name: str) → None[source]

Event handler for service elements in the AndroidManifest.xml.

Parameters:

element – The service element.
name – The name of the service.

property scan: Scan

Returns the scan object associated with the inspector.

Returns:: The Scan object.

SAST Interface

mastf.MASTF.scanners.code.yara_scan_file(file: Path, task: ScanTask, base_dir=None, observer: Observer = None)[source]

Perform YARA scan on a file.

This function performs YARA scan on the specified file using the YARA rules in the given base directory. It creates YaraResult objects for each match found and creates corresponding Snippet and Finding objects to store the scan results.

Parameters:

file – The file path to scan.
task – The ScanTask associated with the scan.
base_dir – The base directory containing the YARA rules.
observer – The observer object for tracking the progress and logging.

Returns:

None

mastf.MASTF.scanners.code.yara_code_analysis(scan_task_pk: str, start_dir: str, observer: Observer = None, base_dir: str = None)[source]

Perform YARA code analysis on files within a directory.

This function performs YARA code analysis on the files within the specified start directory using the provided scan task, base directory, and observer. It scans the files in parallel using multiprocessing or a ThreadPoolExecutor based on the availability of the current process.

Parameters:

scan_task_pk – The primary key of the ScanTask associated with the code analysis.
start_dir – The directory path where the code analysis will be performed.
observer – The observer object for tracking the progress and logging.
base_dir – The base directory containing the YARA rules.

Returns:

None

Usage:

scan_task_pk = "task123"
start_dir = "/path/to/start_directory"
observer = Observer()
base_dir = "/path/to/yara_base_directory"

yara_code_analysis(scan_task_pk, start_dir, observer, base_dir)

mastf.MASTF.scanners.code.sast_scan_file(file_path: Path, task: ScanTask, rules: list) → None[source]

Perform a static application security testing (SAST) scan on a file.

Parameters:

file_path (pathlib.Path) – The path to the file to be scanned.
task (ScanTask) – The scan task associated with the file.
rules (list[pysast.SastRule]) – A list of rules to be used for the scan.

This function performs a SAST scan on the specified file using the provided rules. It creates a new instance of the SAST scanner for each scan to ensure that it accesses the rules’ internal values correctly.

The scan is performed by calling the scan method of the scanner instance and passing the file path as a string argument. If the scan is successful, the function iterates over the scan results and calls the add_finding function to add each finding to the associated scan task.

If an exception occurs during the scan, the error is logged using the global logger instance and the exception is not re-raised.

mastf.MASTF.scanners.code.sast_code_analysis(scan_task: ScanTask, target_dir: Path, observer: Observer, excluded: list, rules_dirs: list) → None[source]

Perform static application security testing (SAST) code analysis on files within a target directory.

This function scans the files within the specified target directory for potential security vulnerabilities using the pySAST library. It applies the provided scan task, rules directories, and exclusion patterns to determine the files to include or exclude from the analysis.

Parameters:

scan_task – The scan task to apply during the code analysis.
target_dir – The directory path where the code analysis will be performed.
observer – The observer object for tracking the progress and logging.
excluded – A list of patterns or regular expressions to exclude specific files or directories from the analysis.
rules_dirs – A list of directories containing the pySAST rules files to use during the analysis.

Raises:

FileNotFoundError – If the target directory does not exist.

Returns:

None

Usage:

scan_task = ScanTask(...)
target_dir = pathlib.Path("/path/to/target/directory")
observer = Observer(...)
excluded = ["txt", "re:test_.*"]
rules_dirs = [pathlib.Path("/path/to/rules/directory")]

sast_code_analysis(scan_task, target_dir, observer, excluded, rules_dirs)

mastf.MASTF.scanners.code.add_finding(match: dict, scan_task: ScanTask) → None[source]

Add a finding to the scan task based on the match information.

This function retrieves the necessary information from the match dictionary to create a finding or vulnerability object and associates it with the provided scan task.

The match dictionary contains information about the finding, such as the internal ID, rule ID, absolute path, lines, and metadata.

First, the function extracts the internal ID from the metadata and tries to find the corresponding :class`FindingTemplate` object in the database. If the template does not exist, an error is logged and the function returns.

The absolute path is converted to a pathlib.Path object, and a Snippet object is created using information from the match dictionary, such as lines, language, file name, and system path.

If the metadata indicates that it is a vulnerability, a Vulnerability object is created with the corresponding template, snippet, scan, scanner, and severity. Otherwise, a Finding object is created.

Parameters:

match (dict) – A dictionary containing the match information.
scan_task (ScanTask) – The scan task to associate the finding with.

class mastf.MASTF.scanners.code.YaraResult(match: dict)[source]

Represents the result of a YARA match.

This class encapsulates the information extracted from a YARA match and provides convenient properties and methods to access and manipulate the match data.

Parameters:: match – The dictionary containing the YARA match data.

get_template() → FindingTemplate[source]

Get the associated finding template.

This method retrieves the associated finding template for the YARA result. It first checks if a template with the specified ID or internal ID exists. If not, it creates a new template using the YARA match metadata.

Returns:: The associated FindingTemplate object, or None if it couldn’t be retrieved or created.

get_template_data() → dict[source]

Get the data for creating a finding template.

This method returns a dictionary containing the data required for creating a finding template based on the YARA match metadata.

Returns:: The data dictionary for creating a finding template.

property internal_id: str

Get the internal ID of the associated finding template.

This property returns the internal ID of the associated finding template from the YARA match metadata.

Returns:: The internal ID of the finding template.

property severity: Severity

Get the severity of the YARA result.

This property returns the severity of the YARA result. It checks the “severity” field in the match metadata and maps it to the corresponding Severity enum value.

Returns:: The Severity enum value representing the severity of the result.

property template_id: str

Get the ID of the associated finding template.

This property returns the ID of the associated finding template from the YARA match metadata.

Returns:: The ID of the finding template.