[o365-plugin] Office 365 collector does not process paginated content-list responses #2436

Acknowledgements

• I have searched (https://github.com/utmstack/UTMStack/issues) for past instances of this issue
• I have verified that my UTMStack version is up-to-date

Describe the bug

The Office 365 collector appears to process only the initial response returned by the Microsoft Office 365 Management Activity API content-list endpoint.

When additional content blobs are available, Microsoft returns a NextPageUri response header. The collector does not appear to read or follow this header, so content blobs listed on subsequent pages may not be downloaded or processed.

This can result in incomplete Microsoft 365 audit-log collection for tenants whose content-list query results are paginated.

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

The Office 365 collector should:

1. Request the initial content list for the selected tenant, subscription, and time range.
2. Process every content blob returned in the response.
3. Check the response for a NextPageUri header.
4. Request and process each subsequent page.
5. Continue until Microsoft no longer returns a NextPageUri header.

All content blobs available for the requested collection window should be processed.

Current Behavior

The collector appears to make one request for the content list and process only the content blobs contained in that response.

The implementation does not appear to check for or follow the NextPageUri response header. When Microsoft paginates the results, content blobs beyond the first page may therefore be omitted.

No application error is necessarily generated because the initial API request can complete successfully while additional pages remain available.

Reproduction Steps

1. Configure the UTMStack Office 365 plugin for a Microsoft 365 tenant.
2. Generate or identify a collection interval with enough Microsoft 365 audit activity for the content-list endpoint to return multiple pages.
3. Query the Microsoft Management Activity API independently for the same tenant, content type, start time, and end time.
4. Confirm that the initial response includes a NextPageUri header.
5. Allow the UTMStack Office 365 collector to process the same interval.
6. Compare the content blob IDs processed by UTMStack with the blob IDs returned across all API pages.
7. Observe that blobs returned after the first page are not processed.

Possible Solution

Update the content-list request logic to return both:

• The response body containing the content blob list.
• The value of the NextPageUri response header, when present.

Continue requesting content-list pages until NextPageUri is absent.

Additional Information/Context

No response

UTMStack Version

v11

Operating System and version

Ubuntu 24.4

Hypervisor and Version | Server Vendor and Model

ESX

Browser and version

Chrome