Hello everyone!

In the previous article, you learned how to manage subdomains in SubScope, including adding, listing, and deleting them. Now, we will delve into the next crucial step: adding URLs to your database.

Based on the help page, to perform domain operations (add, list, and delete), we use the url keyword.

./subscope.py url -h

To work with URL, the following setup is required.

If you’re unable to create the necessary tables with SubScope, I recommend reading the three articles below for guidance.

Program Management

Streamlining Your Bug Bounty Projects: Adding and Managing Programs via SubScope

Domain Management

Effective Domain Management for Bug Bounty Programs Using SubScope

Subdomain Management

A Deep Dive into Subdomain Management Using SubScope

Adding a URL

Based on the picture below, I want to add the following URLs with different options into the SubScope tool.

./subscope.py url add https://engage.tesla.com engage.tesla.com tesla.com tesla_wk --scheme https --status_code 200 --ip 65.9.95.40 --cname dtw7dea2t4fea.cloudfront --webtech "Amazon CloudFront, Nginx" --content_length 180480
./subscope.py url add https://epcapi.tesla.com epcapi.tesla.com tesla.com tesla_wk --scheme https --status_code 404 --ip 104.66.96.56 --cname epcapi.tesla.com.edgekey.net --webtech "Akamai, Varnish"
./subscope.py url add https://exec.teslamotors.com exec.teslamotors.com teslamotors.com tesla_wk --scheme https --status_code 200 --ip 172.233.159.31 --webtech Nginx --content_length 7029 --title Login
./subscope.py url add https://click.comms.walmart.com click.comms.walmart.com walmart.com walmart_wk --scheme https --status_code 403 --webtech HSTS --content_length 1233
./subscope.py url add https://cert-customcards.walmart.com cert-customcards.walmart.com walmart.com walmart_wk --scheme https --status_code 200 --ip 23.2.218.203 --content_length 1719 --title "Program Manager" 

We are adding five URLs to the database. There are many options available for specifying a URL, and below is a list of all the options that can be used.

• --scheme: Scheme (http or https)
• --method: HTTP method (GET, POST, …)
• --port: Port number (80, 443, …)
• --status_code: HTTP status code (200, 404, …)
• --scope: Scope (inscope or outscope)
• --ip: IP address
• --cdn_status: CDN status (yes or no)
• --cdn_name: Name of the CDN provider
• --title: Title of the url
• --webserver: Web server type
• --webtech: Web technologies (comma-separated)
• --cname: CNAME of the live subdomain
• --location: Redirect location
• --flag: Specify a flag for the URL (blank, login, default_page)
• --path: Path of the URL
• --content_length: Content length

Now, we want to make some changes to the added URLs.

• Added IP address and title to https://click.comms.walmart.com

./subscope.py url add https://click.comms.walmart.com click.comms.walmart.com walmart.com walmart_wk --ip 13.111.151.219 --title "403 - Forbidden: Access is denied."

As you can see, the log indicates that only two options were updated. You can use this feature for continuous reconnaissance.

• Add flag to https://exec.teslamotors.com

./subscope.py url add https://exec.teslamotors.com exec.teslamotors.com teslamotors.com tesla_wk --flag login_page 

Note: Every change you make will update the updated_at column accordingly.

Listing URLs

I want to list information about a specific URL:

./subscope.py url list https://click.comms.walmart.com click.comms.walmart.com walmart.com walmart_wk  

To list all URLs associated with tesla.com domain, we can use the following command:

./subscope.py url list \* \* tesla.com tesla_wk

Just like what we learned previously, we have several options for filtering the output, including --scheme, --scope, --method, --port, --status_code,--ip ,--cdn_status , --title , --cdn_name , --title , --webserver , --webtech , --cname , --flag --path and --content_length .

For example, I want to list all URLs with nginx webtech:

./subscope.py url list \* \* \* \* --webserver nginx

or for example, all URLs with cloudfront cname

./subscope.py url list \* \* \* \* --cname cloud

Note: You can input only part of a string, and the database will detect and match it.

Similar to other listing, we also have the --brief and --count options. When you use a filter, these options work as well and return values based on the applied filter.

./subscope.py url list \* \* \* \* --status_code 200 --brief
./subscope.py url list \* \* \* \* --flag login_page --brief

./subscope.py url list \* \* \* \* --count
./subscope.py url list \* \* \* walmart_wk --count

Listing URLs based on timestamp

We have two columns: created_at and updated_at. The created_at column shows the timestamp when the row was inserted into the database, while the updated_at column indicates the timestamp of the last changes made to the row, such as updates to the IP address, scope, status_code, and so on.

We have the ability to search for URLs based on the above columns using --create_time and --update_time. for this demonstration, we will use --update_time, but keep in mind that all actions on this filter also apply to --create_time.

This method is similar to subdomain listing. To review that section, click here.

Statistical Filters

We have included some statistical options to provide an overview of your assets.

./subscope.py url list -h | grep "statistics"

Let’s use some of this options:

./subscope.py url list \* \* \* \* --stats-status-code
./subscope.py url list \* \* \* \* --stats-cname
./subscope.py url list \* \* \* \* --stats-flag

Note: When you use a filter, these options also work and return values based on the applied filter.

Deleting a URL

To delete a specific URL from a domain in a program, you can use the following command:

./subscope.py url delete https://epcapi.tesla.com epcapi.tesla.com tesla.com tesla_wk 

Note: You can delete all URLs based on various options, such as --scheme, --scope, --method, --port, --status_code, --ip, --cdn_status, --title, --cdn_name, --webserver, --webtech, --cname, --flag, --path, and --content_length.

Thank you for following along in this article. We hope this guide has provided valuable insights into managing your bug bounty URLs effectively. With the knowledge you’ve gained, you can now confidently add, list, and delete URLs. Stay tuned for the next sections, where we will delve deeper into adding IPs and enhancing your overall experience with this tool.

Happy bug hunting!

Hello everyone!

In the previous article, you learned how to manage domains in SubScope, including adding, listing, and deleting them. Now, we will delve into the next crucial step: adding subdomains to your domains in a program.

Based on the help page, to perform domain operations (add, list, and delete), we use the subdomain keyword.

./subscope.py sudomain -h

To work with subdomains, the following setup is required.

If you’re unable to create the necessary tables with SubScope, I recommend reading the two articles below for guidance.

Program Management

Streamlining Your Bug Bounty Projects: Adding and Managing Programs via SubScope

Domain Management

Effective Domain Management for Bug Bounty Programs Using SubScope

Adding a Subdomain

For adding subdomain for tesla.com in tesla_wk program, we can use following command:

./subscope.py subdomain add engage.tesla.com tesla.com tesla_wk

Look at the output above — each subdomain has the following options:

• Source: This indicates where the subdomain was discovered, such as crt.sh, subfinder, etc. You can specify one or multiple sources using the --source <source_1> <source_2>option.
• Scope: The scope can either be inscope or outscope. You can define it using the --scope option.
• Resolved: If a subdomain has been resolved, this option is set to yes; otherwise, it is set to no. You can specify the resolving status using the --resolved option.
• IP Address: You can assign an IP address to a subdomain using the --ip option.
• CDN Status: The IP address may or may not belong to a CDN (Content Delivery Network) range. If the IP address is within the range of a CDN (e.g., Cloudflare), set this option to yes; otherwise, set it to no. Use the --cdn_status option to specify this.
• CDN Name: If the IP address is part of a CDN range, you can specify the CDN’s name using the --cdn_name option.

Let’s update the subdomain mentioned above and add some options to it:

./subscope.py subdomain add engage.tesla.com tesla.com tesla_wk --source crtsh subfinder --ip 65.9.95.40 --cdn_status yes --cdn_name cloudfront

Based on the example above, I will add some subdomains:

./subscope.py subdomain add epcapi.tesla.com tesla.com tesla_wk --scope outscope --source subfinder --ip 104.66.96.56 --cdn_status yes --cdn_name akamai
./subscope.py subdomain add exec.teslamotors.com teslamotors.com tesla_wk --source subfinder --ip 172.233.159.31
./subscope.py subdomain add click.comms.walmart.com walmart.com walmart_wk --scope outscope --source crtsh --ip 13.111.151.219 --cdn_status yes --cdn_name akamai
./subscope.py subdomain add cert-customcards.walmart.com walmart.com walmart_wk --source crtsh subfinder --ip 23.2.218.203 --cdn_status yes --cdn_name akamai

It’s common for some options of a subdomain to change over time. For example, the IP address of epcapi.tesla.com may have changed. the syntax for updating a subdomain is the same as for adding a subdomain.

./subscope.py subdomain add epcapi.tesla.com tesla.com tesla_wk --scope outscope --source subfinder --ip 172.233.159.31 --cdn_status no --uncdn_name

If you look at the example above, I used the --uncdn_name option to remove the CDN name. Similarly, you can do the same for IP addresses (--unip) and sources (--unsource <source>).

Listing Subdomains

We have added five subdomains to database based on following scheme:

I want to list information about a specific subdomain:

./subscope.py subdomain list engage.tesla.com tesla.com tesla_wk

To list all subdomains associated with tesla.com domain, we can use the following command:

./subscope.py subdomain list \* tesla.com tesla_wk

To list all subdomains for all domains in the tesla_wk program, use the following command:

./subscope.py subdomain list \* \* tesla_wk

You might guess that to list all subdomains in the database, you can use the following command:

./subscope.py subdomain list \* \* \*

Just like what we learned previously, we have several options for filtering the output, including --source, --scope, --resolved, --cdn_status, --cdn_name, and --ip.

For example, I want to list all in-scope subdomains without any CDN IPs:

./subscope.py subdomain list \* \* \* --scope inscope --cdn_status no

Or, I want to list all subdomains found via the crtsh source:

./subscope.py subdomain list \* walmart.com walmart_wk --source crtsh

We have included an interesting option — --source-only—for explicitly tracking subdomains gathered from a specific source. For example, in the previous command, we found two subdomains identified by crt.sh. Now, I want to display only the subdomains that were found by crtsh.

./subscope.py subdomain list \* walmart.com walmart_wk --source crtsh --source-only

Similar to domain listing, we also have the --brief and --count options. When you use a filter, these options work as well and return values based on the applied filter.

./subscope.py subdomain list \* \* \* --brief
./subscope.py subdomain list \* \* \* --count

./subscope.py subdomain list \* walmart.com walmart_wk --source crtsh --brief
./subscope.py subdomain list \* walmart.com walmart_wk --source crtsh  --count

Listing subdomains based on timestamp

We have two columns: created_at and updated_at. The created_at column shows the timestamp when the row was inserted into the database, while the updated_at column indicates the timestamp of the last changes made to the row, such as updates to the IP address, source, and so on.

We have the ability to search for subdomains based on the above columns using --create_time and --update_time. for this demonstration, we will use --update_time, but keep in mind that all actions on this filter also apply to --create_time.

First, let’s list all subdomains in tesla_wk that were updated in the year 2024. :)

./subscope.py subdomain list \* \* tesla_wk --update_time 2024

Let’s narrow the filter and list all subdomains updated at 2024-10-11–14:54

./subscope.py subdomain list \* \* \* --update_time 2024-10-11-14:54

You can also apply a filter between two timestamps.

./subscope.py subdomain list \* \* \* --update_time 2024-10-11-14:37,2024-10-11-14:50

Statistical Filters

We have included some statistical options to provide an overview of your assets.

./subscope.py subdomain list -h | grep "statistics"

Let’s use some of this options:

./subscope.py subdomain list \* \* \* --stats-scope
./subscope.py subdomain list \* \* \* --stats-cdn-status
./subscope.py subdomain list \* \* \* --stats-source

Note: When you use a filter, these options also work and return values based on the applied filter.

./subscope.py subdomain list \* \* \* --source crtsh --stats-cdn-name

Deleting a subdomain

To delete a specific subdomain from a domain in a program, you can use the following command:

./subscope.py subdomain delete cert-customcards.walmart.com walmart.com walmart_wk

To delete out-of-scope subdomains where their IP range falls within Akamai’s range, you can use the following command:

./subscope.py subdomain delete \* \* \* --scope outscope --cdn_name akamai

Thank you for following along in this article. We hope this guide has provided valuable insights into managing your bug bounty subdomains effectively. With the knowledge you’ve gained, you can now confidently add, list, and delete subdomains. Stay tuned for the next sections, where we will delve deeper into adding URLs (live subdomains) and enhancing your overall experience with this tool.

Happy bug hunting!

In the previous part, you learned how to manage programs in SubScope, including adding, listing, and deleting them. Now, we will delve into the next crucial step: adding domains to your programs.

Based on the help page, to perform domain operations (add, list, and delete), we use the domain keyword.

./subscope.py domain -h

As a reminder, in the previous article, we added two programs: tesla_wk and walmart_wk

Streamlining Your Bug Bounty Projects: Adding and Managing Programs via SubScope

Adding domain

I want to add three domains to the tesla_wk program

./subscope.py domain add tesla.com tesla_wk
./subscope.py domain add teslamotors.com tesla_wk
./subscope.py domain add ts.la tesla_wk

Sometimes, you may want to add an out-of-scope domain to your program. In this case, you can use the --scope outscope option to add it. for example I want to add teslaenergy.services to tesla_wk program

./subscope.py domain add teslaenergy.services tesla_wk --scope outscope

You may have noticed that the --scope inscope option exists. This option is the default when you do not specify a scope in the command.

Now, I want to change the scope of one of the domains:

./subscope.py domain add ts.la tesla_wk --scope outscope 

As you can see, the syntax for changing the scope is exactly the same as that for adding a domain

Listing domains

We have added four domains to the tesla_wk program. Now, I want to list information about a specific domain.

./subscope.py domain list tesla.com tesla_wk

To list all domains associated with tesla_wk, we can use the following command:

./subscope.py domain list \* tesla_wk

Why do the timestamps for the created_at and updated_at columns differ for ts.la? The timestamps differ because we changed the scope of ts.la. The updated_at timestamp reflects the time when this action was performed, which is useful for tracking changes.

If you want to list the in-scope domains, you can use the --scope inscope option to list these domains.

./subscope.py domain list \* tesla_wk --scope inscope

To list out-of-scope domains, you can use the --scope outscope option.

As a reminder, in the program section, we have two options: --brief for listing only the names of the domains, and --count for returning the total count of domains.

Note: When you use a filter (in this case, --scope), these options also work and return values based on the applied filter.

./subscope.py domain list \* tesla_wk --brief
./subscope.py domain list \* tesla_wk --count

./subscope.py domain list \* tesla_wk --scope inscope --brief
./subscope.py domain list \* tesla_wk --scope inscope --count

You can perform above actions on all domains across all programs by simply using an asterisk (*) instead of tesla_wk.

./subscope.py domain list \* \* --scope inscope --brief

Deleting a domain

To delete a specific domain from a program, you can use the following command:

./subscope.py domain delete ts.la tesla_wk

To delete out-of-scope domains from the tesla_wk program, you can use the following command:

./subscope.py domain delete \* tesla_wk --scope outscope

You can perform the above actions on all domains across all programs by using an asterisk (*) instead of tesla_wk. The following command will delete all domains from all programs:

./subscope.py domain delete \* \*

Thank you for following along in this article. We hope this guide has provided valuable insights into managing your bug bounty domains effectively. With the knowledge you’ve gained, you can now confidently add, list, and delete domains. Stay tuned for the next sections, where we will delve deeper into adding subdomains and enhancing your overall experience with this tool.

Happy bug hunting!

I’m excited to introduce a tool I’ve developed for managing domains, subdomains, URLs, and CIDRs. This tool is inspired by the BBRF (Bug Bounty Reconnaissance Framework) and aims to be a more comprehensive solution for reconnaissance and domain management.

This article is divided into multiple sections to cover all aspects of the tool. In this part, we will focus on downloading the tool, adding a program, listing programs, and deleting programs. Let’s get started.

To download this tool, visit my GitHub repository here and clone the project.

git clone https://github.com/hunThubSpace/SubScope.git && cd SubScope

You can run the tool with Python, or give it executable permission using chmod.

chmod +x subscope.py

If you’re using this tool for the first time, it’s recommended to run the application without any parameters. This will initialize and create the database.

./subscope.py

Note: If the database has already been created, the above command will not create it again.

Let’s take a look at the tool’s help page.

./subscope.py -h

Based on the help page, to perform program operations (add, list, and delete), we use the program keyword.

./subscope.py program -h

So, what is a program?

A program can refer to a name, such as a company name, that is used to organize your bug bounty projects. For example, if you want to work on two programs in bug bounty — Tesla and Walmart — each with its own domains, subdomains, and other resources, you can separate them by creating distinct programs.

Adding Program

For this demo, we will add two programs: tesla_wk and walmart_wk.

./subscope.py program add tesla_wk
./subscope.py program add walmart_wk

Note: If you attempt to add a duplicate program, the script will generate an error.

./subscope.py program add tesla_wk

Listing programs

Now that we have added two programs, you can use the following command to display information about a program:

./subscope.py program list tesla_wk

If you use an asterisk (*) instead of a program name, you can list information about all programs.

./subscope.py program list \*

We have included the --brief option to display only the program names.

./subscope.py program list \* --brief

At times, you might need to know the total number of programs. To accommodate this, we have included an additional option: --count, which provides the count of all programs.

./subscope.py program list \* --count

Deleting a program

As you may recall, we have two programs. To delete a program, you can use the following command:

./subscope.py program delete tesla_wk

If you want to delete all programs, use an asterisk (*) instead of a program name like listing subdomains:

./subscope.py program delete \*

I have deleted all programs. To continue the tutorial, I will add two programs again. :)

I have included an option, --all, for deleting all resources associated with a program, such as domains, subdomains, and more.

./subscope.py program delete tesla_wk --all

Why did we include this option?

It’s common to want to delete a program along with all its associated resources. For example, if you worked on the Tesla program in bug bounty but no longer wish to continue, you can use this option to delete the program and all related resources.

If you want to wipe your database of all entries, you can use the following command to delete all programs.

./subscope.py delete program \* --all

Thank you for following along in this article. We hope this guide has provided valuable insights into managing your bug bounty programs effectively. With the knowledge you’ve gained, you can now confidently add, list, and delete programs. Stay tuned for the next sections, where we will delve deeper into adding domains and enhancing your overall experience with this tool.

Happy bug hunting!