Wednesday, December 25, 2013

Filter search results by managed metadata in KQL in Sharepoint 2013

Suppose that we have custom Sharepoint list with custom content type bound to it (let’s call this CT Book). In our CT there is managed metadata field Country, used for targeting books for particular country in search results. Country term set may be hierarchical and look like this:

   1: APAC:
   2:     China
   3:     ...
   4: EMEA:
   5:     Germany
   6:     France
   7:     Russia
   8:     Spain
   9:     UK
  10:     ...
  11: North America:
  12:     Canada
  13:     USA
  14: ...

We want to show books targeted for China only on Chinese site, for Germany on German, etc. How to do it with Sharepoint?

In order to do it we will use KQL. We may define appropriate search query rules directly in Content by search web parts, but it will be needed to be done for each web part separately. Instead we will use another approach: we will create custom search result source and will configure query rules there. With this approach we will need to do it only once and search web parts will use it by default. In one of the previous posts I showed how to create custom search result source: Problem with missing catalog connections when use customized search result source in Sharepoint 2013.

In order to filter results by appropriate country we need to use the following KQL:

   1:  
   2: {?{searchTerms} -ContentClass=urn:content-class:SPSPeople
   3: (ContentType<>Book OR (ContentType=Book AND (Countries:"EMEA:Germany" OR
   4: Countries="EMEA")))}

With this query search results will only show those books which are targeted to Germany or to all EMEA countries.

Configure content by search web parts in Sharepoint 2013 to not use client language

Content by search web parts is new OTB way in Sharepoint 2013 to show content from search index. They have a lot of extensibility points, which make them useful in real-world applications (e.g. ability to use custom web templates or search results source). One problem which we found during working on search-driven site, is that by default they use client browser language for showing search results. I.e. we tested search results with the same search query rules, but with different browser languages (i.e. different Accept-Language http header), and results were different. Some time ago I already wrote about similar problem caused by using browser language for parsing dates in search queries (see Problem in KQL with date times and client object model in Sharepoint). It shows that search-related functionality in Sharepoint 2013 is heavily tight with client language because of some reason. And from my point of view this is controversial decision. At least in my practice in all cases it would be more safe and manageable to define on server side what language should be used for showing search results, instead of relying on client settings.

One example from real life: there is one web application with several language site collections. On production server language packs can’t be installed so all site collections are technically English site collections (SPWeb.Language is 1033 for all of them). In order to distinguish them by languages SPWeb.Locale is used (custom functionality uses it for retrieving strings from local resx files). In this case we need to use SPWeb.Locale set on server side for search results, but not client language. How to achieve it?

I analyzed code of Content by search web parts and related components and found that it uses one of the properties inside DataProviderJSON property called FallbackLanguage. By default it is set to –1:

   1: <property name="DataProviderJSON" type="string">{"QueryGroupName":"a5562ef0-f9e6-45be-af08-570afaeb49e1",
   2:     "QueryPropertiesTemplateUrl":"sitesearch://webroot",
   3:     "IgnoreQueryPropertiesTemplateUrl":false,
   4:     "SourceID":"f26d6d09-e031-4f67-a9a0-eef0e2fd108e",
   5:     "SourceName":null,
   6:     "SourceLevel":null,
   7:     "CollapseSpecification":"",
   8:     "QueryTemplate":"{searchboxquery}",
   9:     "FallbackSort":null,
  10:     "FallbackSortJson":"null",
  11:     "RankRules":null,
  12:     "RankRulesJson":"null",
  13:     "AsynchronousResultRetrieval":false,
  14:     "SendContentBeforeQuery":true,
  15:     "BatchClientQuery":true,
  16:     "FallbackLanguage":-1,
  17:     "FallbackRankingModelID":"",
  18:     "EnableStemming":true,
  19:     "EnablePhonetic":false,
  20:     "EnableNicknames":false,
  21:     "EnableInterleaving":false,
  22:     "EnableQueryRules":true,
  23:     "EnableOrderingHitHighlightedProperty":false,
  24:     "HitHighlightedMultivaluePropertyLimit":-1,
  25:     "IgnoreContextualScope":true,
  26:     "ScopeResultsToCurrentSite":false,
  27:     "TrimDuplicates":true,
  28:     "Properties":{"TryCache":true,
  29:         "Scope":"{Site.URL}",
  30:         "ListId":"19defc1a-a3fb-4c2a-b694-7606de190fee",
  31:         "ListItemId":2,
  32:         "UpdateLinksForCatalogItems":true,
  33:         "EnableStacking":true},
  34:     "PropertiesJson":"{\"TryCache\":true,
  35:         \"Scope\":\"{Site.URL}\",
  36:         \"ListId\":\"19defc1a-a3fb-4c2a-b694-7606de190fee\",
  37:         \"ListItemId\":2,
  38:         \"UpdateLinksForCatalogItems\":true,
  39:         \"EnableStacking\":true}",
  40:     "ClientType":"ContentSearchRegular",
  41:     "UpdateAjaxNavigate":true,
  42:     "SummaryLength":180,
  43:     "DesiredSnippetLength":90,
  44:     "PersonalizedQuery":false,
  45:     "FallbackRefinementFilters":null,
  46:     "IgnoreStaleServerQuery":false,
  47:     "RenderTemplateId":"",
  48:     "AlternateErrorMessage":null,
  49:     "Title":""}
  50: </property>

This is the key for configuring Content by search web parts to not use client language: instead of –1 you should specify locale id, which you would like to use for search results by this Content by search web part. E.g. for Russian sites you should use 1049, for German – 1031, etc.

If you already have provisioned web parts on the living site you can modify them by the following PowerShell script:

   1: param( 
   2:     [string]$siteColl,
   3:     [int]$lcid
   4: )
   5:  
   6: function Fix-Content-By-Search-Web-Part($wp, $webPartManager)
   7: {
   8:     Write-Host "Change fallback language for web part" $wp.Title "to $lcid"
   9: -foregroundcolor green
  10:     if ($wp.DataProviderJSON.Contains(",`"FallbackLanguage`":-1,"))
  11:     {
  12:         $wp.DataProviderJSON =
  13: $wp.DataProviderJSON.Replace(",`"FallbackLanguage`":-1,", ",`"FallbackLanguage`":$lcid,")
  14:         $webPartManager.SaveChanges($wp)
  15:         Write-Host "Fallback language was successfully changed" -foregroundcolor green
  16:     }
  17:     else
  18:     {
  19:         Write-Host "Fallback language not found in DataProviderJSON. Web part will be skipped."
  20: -foregroundcolor yellow
  21:     }
  22: }
  23:  
  24: function Fix-Content-By-Search-Web-Parts-On-Page($item)
  25: {
  26:     $file = $item.File
  27:     Write-Host "Check web parts on page" $file.Url -foregroundcolor green
  28:  
  29:     $webPartManager = $file.Web.GetLimitedWebPartManager($file.Url,
  30: [System.Web.UI.WebControls.WebParts.PersonalizationScope]::Shared)
  31:     $contains = $false
  32:     foreach ($wp in $webPartManager.WebParts)
  33:     {
  34:         if ($wp.GetType().FullName -eq
  35: "Microsoft.Office.Server.Search.WebControls.ContentBySearchWebPart" -and
  36: $wp.DataProviderJSON.Contains(",`"FallbackLanguage`":-1,"))
  37:         {
  38:             $contains = $true
  39:             break
  40:         }
  41:     }
  42:  
  43:     if (-not $contains)
  44:     {
  45:         Write-Host "Page" $file.Url "doesn't contain content by search web parts which should be fixed"
  46: -foregroundcolor green
  47:         return
  48:     }
  49:  
  50:     $shouldBePublished = $false
  51:     if ($file.Level -eq [Microsoft.SharePoint.SPFileLevel]::Published)
  52:     {
  53:         $shouldBePublished = $true
  54:     }
  55:  
  56:     if ($file.CheckOutType -ne [Microsoft.SharePoint.SPFile+SPCheckOutType]::None)
  57:     {
  58:        Write-Host "Undo checkout for page" $file.Url -foregroundcolor yellow
  59:        $file.UndoCheckOut()
  60:     }
  61:     $file.CheckOut()
  62:     # reinstantiate web part manager after file was checked out in order
  63:     # to avoid File is not checked out error
  64:     $webPartManager = $file.Web.GetLimitedWebPartManager($file.Url,
  65: [System.Web.UI.WebControls.WebParts.PersonalizationScope]::Shared)
  66:     $webParts = @()
  67:     foreach ($wp in $webPartManager.WebParts)
  68:     {
  69:         Write-Host "Found web part" $wp.GetType().Name
  70:         if ($wp.GetType().FullName -eq
  71: "Microsoft.Office.Server.Search.WebControls.ContentBySearchWebPart"
  72:         {
  73:             $webParts += $wp
  74:         }
  75:     }
  76:  
  77:     if ($webParts.Count -eq 0)
  78:     {
  79:         return
  80:     }
  81:  
  82:     $webParts | ForEach-Object { Fix-Content-By-Search-Web-Part $_ $webPartManager }
  83:  
  84:     Write-Host "Update and checkin page" $file.Url -foregroundcolor green
  85:     $file.Update()
  86:     $file.CheckIn("Change fallback language for content by search web parts to $lcid.")
  87:     if ($shouldBePublished)
  88:     {
  89:         Write-Host "Publish page" $file.Url -foregroundcolor green
  90:         $file.Publish("Change fallback language for content by search web parts to $lcid.")
  91:         if ($file.DocumentLibrary.EnableModeration)
  92:         {
  93:             $file.Approve("Change fallback language for content by search web parts to $lcid.")
  94:         }
  95:     }
  96: }
  97:  
  98: function Fix-Content-By-Search-Web-Parts-In-Web($w)
  99: {
 100:     Write-Host "Check pages on web" $w.Url -foregroundcolor green
 101:  
 102:     $pw = [Microsoft.SharePoint.Publishing.PublishingWeb]::GetPublishingWeb($w)
 103:     $pagesList = $pw.PagesList
 104:     $pagesList.Items | ForEach-Object { Fix-Content-By-Search-Web-Parts-On-Page $_ }
 105: }
 106:  
 107: if (-not $siteColl)
 108: {
 109:     Write-Host "Specify site collection url in siteColl parameter" -foregroundcolor red
 110:     return
 111: }
 112:  
 113: if (-not $lcid)
 114: {
 115:     Write-Host "Specify target locale id (integer) in lcid parameter" -foregroundcolor red
 116:     return
 117: }
 118:  
 119: $site = Get-SPSite $siteColl
 120: $site.AllWebs | ForEach-Object { Fix-Content-By-Search-Web-Parts-In-Web $_ }

It iterates through all sub sites in site collection, in each sub site it goes through all publishing pages and on each page it finds all Content by search web parts which have FallbackLanguage = -1 and changes them to specified locale id. After applying this script your Content by search web parts will use specified language for search results. Hope that it will help someone.