Mar 29, 2011

SharePoint Large List Performance and Views Performance on Large List


SharePoint is a Collaboration and Content Management tool, although more often gets over boarded with a feel of a true Document Repository tool. SharePoint List/View customization can facilitate storing large number of documents efficiently and behave in line with repository tools, to take this further we can integrate archiving tools with SharePoint and reduce the SQL database load/cost.

Sooner or Later the number of items in most of the out of box lists will outgrow the logical performance threshold of 2000 items per container; this will directly impact the performance of the list and views. SharePoint List design and customization can optimize the list / views performance and efficiently manage the large number of items in the list,

List / Document Library

       · Custom List Design / Provisioning

Instead of having one large list or list with folders, where possible create as many lists as possible based on: Content Type, Logical Category, and so on

o No more than 2000 lists per site (soft limit)

o List is not a RDBMS

o Lookup columns should be used sporadically

· Active List and Archive List

One model, which can be used to have performance intact for list is to have a separate list for archived items. In other words, Items should be archived to another list when the list size approaches 2000-4000.

o Active/Custom List items (maximum 2k-4k items)

Items fall in the active list if

· used almost on daily basis

· in dynamic/transient state

· in draft mode

· recently created

· recently published

· versioning is enabled

o Archive List items (maximum 5 million)

Items based on a pre-configured logic, gets moved from Active List to Archive List

· Archive List should have folder implementation in place to organize the items. Where each folder has maximum of 2000 items and the list will contain total of 5 million items maximum.

· One level deep folder is being recommended, further folder nesting has same issue as file system

· Folder creation scan be based on a logical nomenclature (like year and week number) or a unique id (like guid or date time), so on.

· No versioning required as item is in its final state and archived.
 Views

o Views for Archive list

o It’s advisable to use “Search” feature to lookup an item or desired set of items. This would require SharePoint Search configuration and indexing.

o The other option, which can be leveraged is to view items in the folder by using the corresponding folder link

o Views for Active List i.e. Custom List

o Views created on Lists should be displaying items in batches of 10, 20, not more than 100 items.

o Views filtering should be designed carefully

o Usage of indexed columns

· Each column index consumes disk space and resources in the database. Add indexes only to columns that will be used actively in the views.

· Only one indexed column can be used in a view filter and it’s important that the first column that you use to filter the view has an index

o Filtering by an indexed column is more efficient then defining an item limit in a view.

o Complex filtering should be avoided instead design document library as separate entities

o Indexed column is not utilized in Views that use an OR filter condition.

o View should use a filter that will not touch more than 2000-4000 items during its processing.

o Views designed using SharePoint Designer

o The page size (i.e. downloaded data by the browser to render the page), where views is being designed should be of reasonable size (300-500 KB)

o Data View web part uses web services to fetch data from the list

o Content Query web part is the fastest means of pulling data from list.
 Page

Before making a customized page live in production performance should be analyzed using tools like

o Http watch

o Page speed
 Browser

IE 8.0 is recommended for better performance as compared to IE 6.0 / IE 7.0
IE 8.0 allows more TCP connections at a time compared to IE 6/7

Archiving

o Tools or BLOB storage can be used to reduce the data base load / cost and further enhance the list performance and also bring the archiving capability to its best.

o Records Management can be used to distribute the database Load

Pages