| 2016-05-05
      
      § | 
    
  | 18:17 | <csteipp> | deployed revert/new patches for core & extension for T129506 | [production] | 
            
  | 17:40 | <_joe_> | restarted ocg after packet capture | [production] | 
            
  | 17:40 | <ejegg> | updated payments-wiki from 8962b558b5eb5a4dba91be7ed7649b532fa4ee35 to b16c3cb0e12aacb49b91974d693c401dcaa9cca9 | [production] | 
            
  | 17:36 | <_joe_> | temporarily stopping ocg on ocg1003 to better debug cleanup cache | [production] | 
            
  | 17:23 | <urandom> | Rolling restart of cassandra-metrics-collector in RESTBase test cluster : T134016 | [production] | 
            
  | 17:19 | <urandom> | Rolling restart of cassandra-metrics-collector in RESTBase cluster : T134016 | [production] | 
            
  | 17:09 | <_joe_> | clearing ocg cache entries for ocg1003 | [production] | 
            
  | 17:06 | <oblivian@palladium> | conftool action : set/pooled=no; selector: name=ocg1003.* | [production] | 
            
  | 16:55 | <godog> | restart cassandra-metrics-collector on restbase2001 to test white/black listing | [production] | 
            
  | 16:31 | <mdholloway> | mobileapps finished no-op test deploy | [production] | 
            
  | 16:22 | <mdholloway> | mobileapps starting no-op deploy on scb1001 | [production] | 
            
  | 16:01 | <jynus> | restarting db1023 for reimage to jessie | [production] | 
            
  | 15:10 | <anomie@tin> | Synchronized php-1.27.0-wmf.23/extensions/CentralAuth/: SWAT: Use master CentralAuthUser instances when writing [[gerrit:287079]] (duration: 00m 32s) | [production] | 
            
  | 15:05 | <anomie@tin> | Synchronized wmf-config/InitialiseSettings.php: SWAT: Commons: Restrict changetags userright [[gerrit:286522]] (duration: 00m 29s) | [production] | 
            
  | 15:03 | <anomie@tin> | Synchronized wmf-config/InitialiseSettings.php: SWAT: Grant groups from flaggedrevs the patrol rights at test2wiki [[gerrit:287090]] (duration: 00m 47s) | [production] | 
            
  | 15:02 | <jynus> | stopping db1065 for hardware maintenance | [production] | 
            
  | 14:42 | <elukey> | restarted memcached on mc1008 as part of a performance test (T129963) | [production] | 
            
  | 14:39 | <elukey> | memcached on mc1009 is running now with new parameters only available for 1.4.25-2~wmf1 - part of a performance test (T129963) | [production] | 
            
  | 14:28 | <elukey> | memcached 1.4.25-2~wmf1 manually installed on mc1009 as part of a performance test (T129963) | [production] | 
            
  | 14:02 | <andrewbogott> | stopping nova-api on labnet1002 | [production] | 
            
  | 13:55 | <andrewbogott> | downtimed labservices1001, holmium, labcontrol1001 for one hour, disabled puppet as per https://phabricator.wikimedia.org/T128737 | [production] | 
            
  | 13:14 | <akosiaris> | set ganeti2006 as drained (excluded from allocation operations) | [production] | 
            
  | 11:32 | <gehel> | restarting blazegraph on wdqs1001 | [production] | 
            
  | 10:40 | <akosiaris> | rebooted labsdb1004 | [production] | 
            
  | 10:40 | <akosiaris> | fix problems with url_downloader created by PEBKAC | [production] | 
            
  | 10:23 | <jynus> | stopping mysql & backuping db1023 in preparation for reimage | [production] | 
            
  | 09:59 | <jynus> | SET GLOBAL thread_pool_stall_limit = 10; on db1057 | [production] | 
            
  | 09:56 | <jynus> | SET GLOBAL thread_pool_max_threads = 2000; on db1057 | [production] | 
            
  | 09:54 | <elukey> | installed memcached 1.4.25-2~wmf1 manually on mc2009 as part of T129963 | [production] | 
            
  | 09:49 | <mobrovac> | restbase deploy end of 2a3972a | [production] | 
            
  | 09:42 | <jynus> | powercycling mw2027 | [production] | 
            
  | 09:30 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Really depool db1023 for maintenance (duration: 02m 27s) | [production] | 
            
  | 09:26 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Depool db1023 for maintenance (duration: 02m 44s) | [production] | 
            
  | 09:22 | <_joe_> | reloaded the backend varnsih config on cp1058 | [production] | 
            
  | 09:22 | <mobrovac> | restbase deploy start of 2a3972a on canary restbase1008 | [production] | 
            
  | 08:57 | <hoo> | Restarted blazegraph on wdqs1002 (unresponsive) | [production] | 
            
  | 08:12 | <hoo> | Restarted blazegraph on wdqs1002 (unresponsive) | [production] | 
            
  | 05:52 | <gehel> | restarting elasticsearch server elastic1008.eqiad.wmnet (T110236) | [production] | 
            
  | 04:55 | <gehel> | restarting elasticsearch server elastic1007.eqiad.wmnet (T110236) | [production] | 
            
  | 03:27 | <ori> | restarted parsoid on all wtp* hosts to kill any requests from changepropagation service that may have been in flight | [production] | 
            
  | 03:20 | <ori> | ran: sudo salt 'scb*' cmd.run 'puppet agent --disable ; service changeprop stop' | [production] | 
            
  | 03:15 | <ori> | and restarted parsoid on wtp1* | [production] | 
            
  | 03:14 | <ori> | injected an early return to v1Wt2html in routes.js when oldid = 106801025 | [production] | 
            
  | 02:49 | <mwdeploy@tin> | sync-l10n completed (1.27.0-wmf.22) (duration: 18m 23s) | [production] | 
            
  | 00:27 | <twentyafterfour> | phabricator upgrade complete | [production] | 
            
  | 00:21 | <catrope@tin> | Synchronized wmf-config/CommonSettings.php: Plumbing for wmgEchoCrossWikiByDefault (unused for now) (duration: 00m 24s) | [production] | 
            
  | 00:16 | <catrope@tin> | Synchronized wmf-config/InitialiseSettings.php: Plumbing for wmgEchoCrossWikiByDefault (unused for now) (duration: 00m 26s) | [production] | 
            
  | 00:06 | <aude@tin> | rebuilt wikiversions.php and synchronized wikiversions files: Put wikidata on wmf/1.27.0-wmf.23 | [production] | 
            
  | 00:04 | <twentyafterfour> | taking phabricator offline for maintenance | [production] |