Differences

This shows you the differences between two versions of the page.

--- kb:solr:performance_tune [2017/10/25 13:23] – created yehuda
+++ kb:solr:performance_tune [2022/01/03 16:03] (current) – external edit 127.0.0.1
@@ Line 1: / Line 1: @@
 see [[kb:java:preformance]]
+====== Conf ======
+<code>
+Good Day!!
+Thank You for your patience and co-ordination.
+From the below solr logs we suspect that Indexed data in Shard "CALSecurityLogs_shard8_replica2" might have got corrupted
+{{
+-10-22 14:07:04,427 ERROR org.apache.solr.servlet.SolrDispatchFilter: null:org.apache.solr.common.SolrException: SolrCore 'CALSecurityLogs_shard8_replica2' is not available due to init failure: Error opening new searcher
+        at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:783)
+        at org.apache.solr.servlet.SolrDispatchFilter.httpSolrCall(SolrDispatchFilter.java:362)
+        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:260)
+        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:255)
+        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
+        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
+        at org.apache.solr.servlet.SolrHadoopAuthenticationFilter$2.doFilter(SolrHadoopAuthenticationFilter.java:408)
+        at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:622)
+        at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationFilter.doFilter(DelegationTokenAuthenticationFilter.java:291)
+        at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:574)
+        at org.apache.solr.servlet.SolrHadoopAuthenticationFilter.doFilter(SolrHadoopAuthenticationFilter.java:413)
+        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
+        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
+        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
+        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
+        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
+        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
+        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
+        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
+        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:861)
+        at
+Caused by: java.io.EOFException: read past EOF: _b6eo.si
+        at org.apache.solr.store.blockcache.CustomBufferedIndexInput.refill
+}}
+In looking at your solr configs, solr.hdfs.blockcache.write.enabled is set to true, and that would definitely contribute to the issues we are seeing.
+That is noted in the known release issues for Search https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.cloudera.com%2Fdocumentation%2Fenterprise%2Frelease-notes%2Ftopics%2Fcdh_rn_search_ki.html%23known_issues__section_yjg_mbk_ln&data=02%7C01%7Cyehudako%40matrixbi.co.il%7Cb98559ef5ce2481afa1108d51ba1e32d%7C8b3747c3cccd4b178f18bd610edc86e6%7C0%7C0%7C636445304488866387&sdata=tkTDJnz0A4rT0gl%2Bfux7L6lzvQa4ehKAJXZ9UlvtkPI%3D&reserved=0
+It is recommended to turn off blockcache.write
+To do so, you would need to do the following (where collection_name is the name of your:
+. get the current instancedir from ZK:
+solrctl instancedir --get CALSecurityLogs /path/to/collection
+. Change the solr.hdfs.blockcache.write.enabled to false in solrconfig.xml:
+<bool name="solr.hdfs.blockcache.write.enabled">${solr.hdfs.blockcache.write.enabled:false}</bool>
+. upload your instancedir back to ZK after changing the value from true to false:
+solrctl instancedir --update CALSecurityLogs /path/to/collection
+. Reload the solr collection:
+solrctl collection --reload CALSecurityLogs
+. Verify through the Solr UI that the value is changed correctly:
+Go to https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fyour.solr.server%3A8983&data=02%7C01%7Cyehudako%40matrixbi.co.il%7Cb98559ef5ce2481afa1108d51ba1e32d%7C8b3747c3cccd4b178f18bd610edc86e6%7C0%7C0%7C636445304488866387&sdata=NWBYRgwPF2bAxm4ZjGbDmgdFWA5n6o0O7oETrBE8Rsg%3D&reserved=0, select your core, and then select config and verify that solr.hdfs.blockcache.write.enabled is defaulted to false.
+. Restart the solr instances (if you need to do a rolling restart, you can restart each one separately)
+Once that is done, it would be helpful to monitor the logs to confirm there are no further issues.
+=====================
+Enabling blockcache writing may result in unusable indexes
+It is possible to create indexes with solr.hdfs.blockcache.write.enabled set to true. Such indexes may appear corrupt to readers, and reading these indexes may irrecoverably corrupt indexes. Blockcache writing is disabled by default.
+Workaround: Do not enable blockcache writing.
+Bug ID: CDH-17978
+</code>

kb/solr/performance_tune.1508937810.txt.gz · Last modified: 2022/01/03 16:03 (external edit)

Back to top