Hi all,
We've had an unexpected outage on two of our reporting services server overnight and we're hoping someone can point us in the right direction. We have a Windows Server 2008 machine running about 5 instances of reporting services in standalone, non-IIS mode.
Two of these instances have started hitting exceptions when rendering any reports. I include an excerpt from the log below.
The first evidence of a problem in the log is an InternalCatalogException, of which I can find many unrelated hits when searching the forums so I don't think this is specific enough to trace. I do note a reference to "segmentedChunkStore" in the text which seems like a better clue.
I've eliminated the idea of a machine/OS issue due to several instances not affected by this problem. There is some possibly related info which may help:
-This machine is a HyperV VM which has been moved between hosts within the last few weeks. In the process, all Report Server databases were backed up and restored at the new site.
- It uses a Custom Authentication extension authored by us.
- Within the last few days, users of three RS instances advised that a role permission they had prior to the move (Consume Reports Task) was no longer granted. I had to re-grant this to the role. The two
RS instances now experiencing the InternalCatalogException issue raised an error at the time about not being able to find a group when I granted the task. I added the group, granted again, and it granted without error so I thought nothing more
of it.
- These two RS instances have been working fine since the HyperV host move right up until last night.
- The machine had several Windows Updates queued up which were installed last night. Since completion of the updates and subsequent reboot, these two Reporting Services instances now raise this InternalCatalogException error when trying to render any
reports from Report Manager, Report Server, or previous from Report Viewer.
At this point, I'm suspecting some sort of data corruption has occurred in the ReportServer databases, initially causing the loss of the Consume Reports permission and, after a reboot, these exeptions. But any suggestions on how to troubleshoot further greatly appreciated. Here is the excerpt from the log.
library!ReportServer_0-1!8d4!09/19/2012-20:55:53:: Call to GetItemTypeAction(/Risk Manager/Dashboard Module/Medsys/IncConseqCountYTD).
library!ReportServer_0-1!12f0!09/19/2012-20:55:53:: Call to GetItemTypeAction(/Risk Manager/Dashboard Module/Medsys/IncCount13Months).
library!ReportServer_0-1!f98!09/19/2012-20:55:53:: Call to GetItemTypeAction(/Risk Manager/Dashboard Module/Medsys/my30DayActionList).
library!ReportServer_0-1!10c0!09/19/2012-20:55:53:: Call to GetItemTypeAction(/Risk Manager/Dashboard Module/Medsys/org30DayStatusPanel).
library!ReportServer_0-1!830!09/19/2012-20:55:54:: i INFO: RenderForNewSession('/Risk Manager/Dashboard Module/Medsys/org30DayStatusPanel')
library!ReportServer_0-1!830!09/19/2012-20:55:54:: e ERROR: Throwing Microsoft.ReportingServices.Diagnostics.Utilities.InternalCatalogException: An internal error occurred on the report server. See the error log for more details.,segmentedChunkStore;
Info: Microsoft.ReportingServices.Diagnostics.Utilities.InternalCatalogException: An internal error occurred on the report server. See the error log for more details.
library!ReportServer_0-1!830!09/19/2012-20:55:55:: i INFO: Exception InternalCatalogException dumped to: C:\Program Files\Microsoft SQL Server\MSRS10.TESTIMPACRS\Reporting Services\Logfiles flags= ReferencedMemory, AllThreads, SendToWatson
chunks!ReportServer_0-1!830!09/19/2012-20:55:55:: w WARN: Rolling back shared chunk transaction for snapshot '49663ae9-6692-4ea2-8a5d-210de4729653', Permanent=False.
library!ReportServer_0-1!830!09/19/2012-20:55:55:: w WARN: Microsoft.ReportingServices.Diagnostics.Utilities.InternalCatalogException: An internal error occurred on the report server. See the error log for more details.
library!ReportServer_0-1!ff8!09/19/2012-20:55:55:: i INFO: RenderForNewSession('/Risk Manager/Dashboard Module/Medsys/my30DayStatusPanel')
library!ReportServer_0-1!8d4!09/19/2012-20:55:55:: i INFO: RenderForNewSession('/Risk Manager/Dashboard Module/Medsys/triCountPanel')
library!ReportServer_0-1!8d4!09/19/2012-20:55:55:: e ERROR: Throwing Microsoft.ReportingServices.Diagnostics.Utilities.InternalCatalogException: An internal error occurred on the report server. See the error log for more details., segmentedChunkStore;
Info: Microsoft.ReportingServices.Diagnostics.Utilities.InternalCatalogException: An internal error occurred on the report server. See the error log for more details.
library!ReportServer_0-1!ff8!09/19/2012-20:55:55:: e ERROR: Throwing Microsoft.ReportingServices.Diagnostics.Utilities.InternalCatalogException: An internal error occurred on the report server. See the error log for more details., segmentedChunkStore;
Info: Microsoft.ReportingServices.Diagnostics.Utilities.InternalCatalogException: An internal error occurred on the report server. See the error log for more details.
chunks!ReportServer_0-1!ff8!09/19/2012-20:55:55:: w WARN: Rolling back shared chunk transaction for snapshot '68d6456c-1ed0-46eb-8bec-37b04f94065a', Permanent=False.
library!ReportServer_0-1!ff8!09/19/2012-20:55:55:: w WARN: Microsoft.ReportingServices.Diagnostics.Utilities.InternalCatalogException: An internal error occurred on the report server. See the error log for more details.
library!ReportServer_0-1!8d4!09/19/2012-20:55:56:: i INFO: Exception InternalCatalogException dumped to: C:\Program Files\Microsoft SQL Server\MSRS10.TESTIMPACRS\Reporting Services\Logfiles flags= ReferencedMemory, AllThreads, SendToWatson
chunks!ReportServer_0-1!8d4!09/19/2012-20:55:56:: w WARN: Rolling back shared chunk transaction for snapshot '464d6ec5-f256-4633-97d9-792c700931a0', Permanent=False.
library!ReportServer_0-1!8d4!09/19/2012-20:55:56:: w WARN: Microsoft.ReportingServices.Diagnostics.Utilities.InternalCatalogException: An internal error occurred on the report server. See the error log for more details.
library!ReportServer_0-1!b90!09/19/2012-20:55:56:: i INFO: RenderForNewSession('/Risk Manager/Dashboard Module/Medsys/my30DayActionList')
library!ReportServer_0-1!b90!09/19/2012-20:55:56:: e ERROR: Throwing Microsoft.ReportingServices.Diagnostics.Utilities.InternalCatalogException: An internal error occurred on the report server. See the error log for more details., segmentedChunkStore;
Info: Microsoft.ReportingServices.Diagnostics.Utilities.InternalCatalogException: An internal error occurred on the report server. See the error log for more details.
library!ReportServer_0-1!1598!09/19/2012-20:55:57:: i INFO: RenderForNewSession('/Risk Manager/Dashboard Module/Medsys/IncCount13Months')
library!ReportServer_0-1!b90!09/19/2012-20:55:57:: i INFO: Exception InternalCatalogException dumped to: C:\Program Files\Microsoft SQL Server\MSRS10.TESTIMPACRS\Reporting Services\Logfiles flags= ReferencedMemory, AllThreads, SendToWatson
chunks!ReportServer_0-1!b90!09/19/2012-20:55:57:: w WARN: Rolling back shared chunk transaction for snapshot '94bb87a7-bece-454e-b418-78b18b9eacb1', Permanent=False.
library!ReportServer_0-1!b90!09/19/2012-20:55:57:: w WARN: Microsoft.ReportingServices.Diagnostics.Utilities.InternalCatalogException: An internal error occurred on the report server. See the error log for more details.
library!ReportServer_0-1!8d4!09/19/2012-20:55:57:: i INFO: RenderForNewSession('/Risk Manager/Dashboard Module/Medsys/IncConseqCountYTD')
library!ReportServer_0-1!1598!09/19/2012-20:55:57:: e ERROR: Throwing Microsoft.ReportingServices.Diagnostics.Utilities.InternalCatalogException: An internal error occurred on the report server. See the error log for more details., segmentedChunkStore;
Info: Microsoft.ReportingServices.Diagnostics.Utilities.InternalCatalogException: An internal error occurred on the report server. See the error log for more details.
library!ReportServer_0-1!8d4!09/19/2012-20:55:58:: e ERROR: Throwing Microsoft.ReportingServices.Diagnostics.Utilities.InternalCatalogException: An internal error occurred on the report server. See the error log for more details., segmentedChunkStore;
Info: Microsoft.ReportingServices.Diagnostics.Utilities.InternalCatalogException: An internal error occurred on the report server. See the error log for more details.
chunks!ReportServer_0-1!8d4!09/19/2012-20:55:58:: w WARN: Rolling back shared chunk transaction for snapshot 'e254656f-6c22-49e6-b473-5ed986e0a730', Permanent=False.
library!ReportServer_0-1!8d4!09/19/2012-20:55:58:: w WARN: Microsoft.ReportingServices.Diagnostics.Utilities.InternalCatalogException: An internal error occurred on the report server. See the error log for more details.
library!ReportServer_0-1!1598!09/19/2012-20:55:58:: i INFO: Exception InternalCatalogException dumped to: C:\Program Files\Microsoft SQL Server\MSRS10.TESTIMPACRS\Reporting Services\Logfiles flags= ReferencedMemory, AllThreads, SendToWatson
chunks!ReportServer_0-1!1598!09/19/2012-20:55:58:: w WARN: Rolling back shared chunk transaction for snapshot 'e15cc0b9-c8b1-446a-a3bf-da2ceba6273d', Permanent=False.
library!ReportServer_0-1!1598!09/19/2012-20:55:58:: w WARN: Microsoft.ReportingServices.Diagnostics.Utilities.InternalCatalogException: An internal error occurred on the report server. See the error log for more details.
library!WindowsService_0!14a0!09/19/2012-21:01:07:: i INFO: Call to CleanBatch()
library!WindowsService_0!14a0!09/19/2012-21:01:13:: i INFO: Cleaned 0 batch records, 0 policies, 0 sessions, 0 cache entries, 0 snapshots, 0 chunks, 0 running jobs, 0 persisted streams, 0 segments, 0 segment mappings.
Regards,
Michael