Initial file metadata collection scan FAQs

Overview

When you deploy the insider risk agent to user devices, Incydr scans and indexes all files on user devices.

In addition, when you add a cloud data connection, you can optionally enable an inventory of all in-scope drives, which performs a one-time analysis of the sharing state of the files in the in-scope drives.

This article answers frequently asked questions about this process, including:

How long does scanning take?
When can I start viewing file events in the Incydr console?
What is the CPU impact to user devices?

Definitions

Endpoint data source: A single user device running the insider risk agent.
Cloud service data source: A corporate cloud service you authorize Incydr to monitor. Examples include Box, Google Drive, and Microsoft OneDrive for Business. This does not include email data sources (such as Gmail and Office 365 email) because there is no initial file scan for email services.
Initial ingest: The file scan process that indexes all files on an endpoint data source. This scan is performed by the insider risk agent installed on each device.
Initial inventory: The optional file scan process that inventories all files in a cloud data source. The initial inventory data is only collected once and only available for the duration of your data retention period. This scan is performed via a direct connection between Incydr and the cloud service and does not involve agents on user devices. This scan typically begins after a cloud service connection is first authorized and does not need to be complete for Incydr to begin monitoring for user activity in those cloud drives.

FAQs

What is the initial scan?

There are different scans for endpoints and for cloud data connections.

Endpoint data source: The initial ingest scan indexes all files on the device. This ingest creates a record of all files on the device at the time of the scan. As a result, user devices might temporarily use a high percentage of CPU resources. Once the initial scan is complete, Incydr only monitors new and incremental file changes, using significantly fewer CPU resources.
Cloud service data sources: The initial inventory process scans all files in your organization's cloud drives that belong to monitored users. Incydr connects directly to the cloud service to capture this data, and simultaneously starts monitoring file activity right away while completing the inventory process. This scan does not affect user devices, but inventorying all drives consumes significant cloud resources and may take a week or more to complete. In some cases, it may also cause the cloud service to throttle API requests, which can adversely affect the performance of other, non-Incydr activity in the cloud service.
Shared libraries in Microsoft are not inventoried, discovered, or monitored
Incydr can only monitor drives in Microsoft OneDrive. While you can create a shared library within OneDrive, such libraries are actually created as Team Sites in SharePoint. Because Incydr cannot monitor sites in SharePoint, any shared libraries listed in your OneDrive environment are excluded

How long does the scan take?

Endpoint data source: Ingest times vary based on the number and size of files on each device. The processing power of the device also affects how long it takes to complete.
Cloud service data sources: The length of time it takes for the initial inventory to complete depends on the number of files in the drives that belong to the in-scope users who are being monitored in your environment. The inventory process is not a prerequisite to begin monitoring user activity in your cloud storage.
- For environments that contain hundreds of drives, the initial inventory may take between 24 and 72 hours depending on the number of files in each user's drives. The inventory process can take longer if Incydr's connection to the cloud environment is throttled. Throttling may occur for these reasons:
  - Google Drive connections can be throttled based on the number of requests made by both the Incydr service per user drive and by all services in the account as a whole.
  - OneDrive connections can be throttled based on all requests (including those from Incydr) for the account as a whole.
  - Box connections can be throttled based on requests made by Incydr per user drive
- For environments that contain thousands of drives, the initial inventory completes over a longer period. Typically, drives in larger environments complete the inventory process over these time frames:
  - 60% of total drives complete between 24 and 72 hours
  - 25% of total drives complete between 3 and 5 days
  - 15% of total drives complete between 6 and 10 days

Can I start viewing file activity before the scan is fully complete?

Endpoint data sources: Yes. As soon as a file is scanned and indexed, file events for that file are visible in Incydr. In addition, file activity that may indicate an exposure risk (such as moving files to removable media, uploading to personal cloud services or email) are given priority over indexing all files on the device and are reported in near real-time.
Cloud data sources: Yes. Incydr starts monitoring file activity in your organization's environment right away while scanning and completing an inventory of the files on drives owned by in-scope users. File events for these drives become available in Incydr soon after they occur, even if the inventory of those drives has not completed.

How do I know when the scan is complete?

Endpoint data source

Scan status is visible in agent logs for each device:

Sign in to the Incydr console.
Select Administration > Environment > Agents.
Select an agent.
The agent details appear.
Select Actions > Retrieve agent logs, then click Retrieve agent logs.
In the Retrieve log: Email notification dialog, select Yes and enter an email address to receive an email notification, or select No.
Click Apply.
Retrieving... displays under the Logs URL column of the table. (If the device is offline, Retrieving... displays until the device is online.) When the logs are available, a Download logs link displays.
Click the Download logs link.
The logs are downloaded in a ZIP file.
Navigate to the location of the downloaded log archive and open the archive.
Locate and open the service.log.0 file.
Search the service.log.0 file for these strings, which indicate the initial scan is complete:
- Transitioned FFS ingest state from INITIAL_INGEST to SCAN_SUCCESS
- Transitioned FFS ingest state from SCAN_SUCCESS to STEADY_STATE
  If the above strings do not appear in the log file, the scan is still in process, or the scan completed long enough ago that the messages exist in an older version of the log file (for example service.log.1 or service.log.2).
  Contact our Technical Support Engineers if you need help determining the scan status for a device.

Cloud service data connections

In the Incydr console, go to Administration > Integrations > Data Connections and review the Status column.

A status of Monitoring indicates that Incydr is currently monitoring the cloud environment for file activity. If you have just authorized the Incydr connection to the cloud storage environment, Incydr simultaneously completes the inventory process.
To view more detailed status information, click a row in the Data Sources table to open the details panel for that cloud service. This panel lists the total number of monitored users for which Incydr has discovered drives and is currently monitoring for file activity. For Google Drive, a second section repeats these details for shared or team drives.

How much memory does Incydr use?

Incydr agents dynamically set memory allocation to use 25% of the physical memory on the device. For example, if the device has 8GB of RAM, the agent can use up to 2GB.

Search