QUESTION:

We have noticed a few projects that hadn't gotten a rollup which leads me to believe the overnight job is not working correctly

 

REASON:

All projects in the system were included in portfolio regardless of status.

Best practices for creating portfolio is to have a reasonable amount of projects in it. There is a hard coded value of 500 after which Auto-add may not work. Typically customers would filter based on projectstatus = 'A' to include only Active projects because changes mostly happen on active projects. If you have more than 500 active projects you can filter down further based on Project Manager for example.
There will be no changes on the closed (completed) projects so there is no need to keep them in a portfolio. When closing a project, Project manager can rollup the project from the profile page for the final totals.
Similarly review the need to have the projects in draft or inactive status in a portfolio, and if needed create a separate portfolio for these statuses.
Rollup will happen only if there was a change on the project (actual, planned, forecast).

RESOLUTION:

Create new portfolio for Active projects only.

On the Items tab select SQL and enter following select statement:
SELECT projectid, name FROM project WHERE deleted = 0 and projectstatus = 'a'
Check Auto-add and Auto-remove boxes and save without clicking GO.
Overnight job will populate items and roll up the projects that were changed since last rollup.