The defaults for two crawler settings, “Max Pages” and “Max Depth” can be set at the organization level. Once these are set, all new websites created will have this default. Setting new organization crawler defaults will not override any existing individual website’s crawler settings. You can adjust individual websites crawler settings in that website’s settings.
Only users with a role that has the “Settings” permission enabled can adjust organization default crawler settings.
Changing Organization Default Crawler Settings
The following steps will walk you through changing an organization’s default crawler settings.
- In the Main Menu, activate the “Settings” button to navigate to the organization settings.
- On the Organization Settings page, activate the “Organization Defaults” tab.
- Activate the “Crawler Options” link to open crawler settings.
- Change the “Max Pages” and “Max Depth” to be your desired new settings. You may also choose to include subdomains (typically not selected to be an organization default).
- Activate the “Save Button” to save the new organization defaults crawler settings.
All new websites created after this is set will have these settings.