This update includes the following:
- Quick scan/rescan
- Website mass import by csv
- Website bulk crawl and other actions
- Dashboard website filter
- Scan/Crawl Speed settings
- Cancel scan/crawl update
- active crawler widget improvements
- Infrastructure update/decrease scan queue time
Quick scan/rescan
There is now a scan button on the website edit view, the group edit view, and the scan details view. Something simple to make your life a little easier. You can of course still start a scan from the Scan view the same as before this update.
On the website edit screen the Scan button can be found in the top right, it is the first button in the view after the first heading. When activated it will immediately queue the website you are editing to scan.
On the scan details view the Rescan button is in the top right next to the scan status and is the first button on the page after first heading. When activated the website attached to the scan will rescan.
On the group view the Scan button is inside of the edit group widget between the Save and Delete buttons. When activated it will queue all of the websites in the group currently being edited to scan.
The quick scans buttons are only available to users given the, “Manage Scans” permission by their account owner.
Dashboard website filter
The dashboard website filter found at the top right of the dashboard after the first header was updated to be a combo box instead of a dropdown. Groups with dozens or hundreds of websites couldn’t easily find the website they were looking for. With this update the filter can also search websites that are in children groups to make it easier to filter the dashboard to a single website.
Website mass import by csv
We heard you, adding dozens, or hundreds or even thousands of websites was a bit of a pain. For large numbers of websites we would offer to take care of this for you but with this update you can simply upload a csv of your websites from the website view.
The csv has columns for setting the crawl and scan settings for the uploaded websites as well as the parent group. If the parent group set in the csv doesn’t exist it will create the group inside of the root organization group.
This option is only available to users with access to the root organization group.
Website bulk crawl and other actions
From the website view you can now crawl or scan multiple websites. For example to crawl multiple websites you would select the checkbox next to the websites you wanted to crawl and then activated the crawl button.
The header row of the websites listings has a select all checkbox, by selecting this you can select all of the websites on the current page, once selected there is an option to select all of the websites in the entire filter.
Additional filters
We also added 2 additional filters to the website view, you can now filter to view only the websites with no pages or no scans. This can be helpful to find websites that you haven’t crawled or scanned yet.
Scan/Crawl Speed settings
We have added scan speeds to the interface to let you control how fast our scanners will scan your pages. There are 4 options, low, medium, high or manual. The default scan speed is Medium and is 60 pages/minute. This speed can be slowed down or sped up, the manual option lets you specify a custom number of pages per minute.
The website crawl settings have been updated to match the scan speed setting UI. The speeds don’t necessary mean that many pages will be crawled/scanned per minute as there are many factors in play but does mean the speed is limited.
Cancel scan/crawl update
Previously the cancel scan functionality took longer than ideal, it would be sending the cancel request to the already queued pages. This has been updated so a cancel scan typically happens in a few seconds.
Along with this there is also a cancel all queued scans and cancel all queued crawls on the scan view. The cancel all buttons can be found above the listing of crawls/scans.
Infrastructure update/decrease scan queue time
We spent a lot of time to improve the flow for scans, previously scans usually started pretty quickly but sometimes when demand was high they would queue up taking longer to start than ideal.
With this update the platform will automatically spin up additional servers within a few minutes when demand is high so websites will start scanning quicker.