The short-version of the story is that when we run a particular report (a "scorecard" with a fairly extensive number of objects/subreports/etc.) in SSRS 2008 R2 and attempt to have it paginate the scorecard for multiple groupings the report will never finish and eventually throws a timeout error from ReportServer and the process is killed. We’ve isolated the issue to the ReportServer and debugged our application all the way to the handoff point to the Report Viewer specific code. So we know that something is happening in ReportServer that causes the report to never finish executing and can also cause other reports to no longer function. The kicker is that we’ve verified it happens only on servers with the R2 stack and that non-R2 servers are not affected (tested across multiple VMs).
To continue with a little more technical detail, we profiled the ReportServer database and found a process that seemed to be in an endless loop of execution, using up both physical resources and creating a huge number of batches and locks that continued to grow for the entire duration of the profile trace. After allowing the process to continue for just over 30 minutes it had accumulated over 14,000 X-locks, mostly on the same three resources. I've got a spreadsheet with some output of sp_lock and sys.dm_tran_locks if someone is curious; you can see that the process ID in question for this run was 98. The sp_lock tab was updated just before we killed the process, the sys.dm_tran_locks was snapshotted after about 10 minutes of runtime. What’s strange is that when I ran the following:
sp_lock @spid1=98
GO
SELECT*FROMsys.dm_tran_locksWHERE request_session_id= 98
The process in question is creating a ton of locks on three different objects, mostly the first 2.
OBJECT_NAME | object_id |
Segment | 133575514 |
ChunkSegmentMapping | 181575685 |
SegmentedChunk | 245575913 |
From my limited research these tables are the primary “data movers” when reporting services renders a report, so it seems like this is an internal issue within report server.
That being said, I wanted to try posting here to see if anyone else has come across this problem before? I've searched the web and can't seem to find anything similar other than a StackOverflow post that describes the same behavior but appears to be related to a different issue. Before I file a bug with MS I thought I'd check to make sure I wasn't missing something obvious.
Thanks,
Jeremy