| 2016-05-05
      
      § | 
    
  | 16:31 | <mdholloway> | mobileapps finished no-op test deploy | [production] | 
            
  | 16:22 | <mdholloway> | mobileapps starting no-op deploy on scb1001 | [production] | 
            
  | 16:01 | <jynus> | restarting db1023 for reimage to jessie | [production] | 
            
  | 15:10 | <anomie@tin> | Synchronized php-1.27.0-wmf.23/extensions/CentralAuth/: SWAT: Use master CentralAuthUser instances when writing [[gerrit:287079]] (duration: 00m 32s) | [production] | 
            
  | 15:05 | <anomie@tin> | Synchronized wmf-config/InitialiseSettings.php: SWAT: Commons: Restrict changetags userright [[gerrit:286522]] (duration: 00m 29s) | [production] | 
            
  | 15:03 | <anomie@tin> | Synchronized wmf-config/InitialiseSettings.php: SWAT: Grant groups from flaggedrevs the patrol rights at test2wiki [[gerrit:287090]] (duration: 00m 47s) | [production] | 
            
  | 15:02 | <jynus> | stopping db1065 for hardware maintenance | [production] | 
            
  | 14:42 | <elukey> | restarted memcached on mc1008 as part of a performance test (T129963) | [production] | 
            
  | 14:39 | <elukey> | memcached on mc1009 is running now with new parameters only available for 1.4.25-2~wmf1 - part of a performance test (T129963) | [production] | 
            
  | 14:28 | <elukey> | memcached 1.4.25-2~wmf1 manually installed on mc1009 as part of a performance test (T129963) | [production] | 
            
  | 14:02 | <andrewbogott> | stopping nova-api on labnet1002 | [production] | 
            
  | 13:55 | <andrewbogott> | downtimed labservices1001, holmium, labcontrol1001 for one hour, disabled puppet as per https://phabricator.wikimedia.org/T128737 | [production] | 
            
  | 13:14 | <akosiaris> | set ganeti2006 as drained (excluded from allocation operations) | [production] | 
            
  | 11:32 | <gehel> | restarting blazegraph on wdqs1001 | [production] | 
            
  | 10:40 | <akosiaris> | rebooted labsdb1004 | [production] | 
            
  | 10:40 | <akosiaris> | fix problems with url_downloader created by PEBKAC | [production] | 
            
  | 10:23 | <jynus> | stopping mysql & backuping db1023 in preparation for reimage | [production] | 
            
  | 09:59 | <jynus> | SET GLOBAL thread_pool_stall_limit = 10; on db1057 | [production] | 
            
  | 09:56 | <jynus> | SET GLOBAL thread_pool_max_threads = 2000; on db1057 | [production] | 
            
  | 09:54 | <elukey> | installed memcached 1.4.25-2~wmf1 manually on mc2009 as part of T129963 | [production] | 
            
  | 09:49 | <mobrovac> | restbase deploy end of 2a3972a | [production] | 
            
  | 09:42 | <jynus> | powercycling mw2027 | [production] | 
            
  | 09:30 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Really depool db1023 for maintenance (duration: 02m 27s) | [production] | 
            
  | 09:26 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Depool db1023 for maintenance (duration: 02m 44s) | [production] | 
            
  | 09:22 | <_joe_> | reloaded the backend varnsih config on cp1058 | [production] | 
            
  | 09:22 | <mobrovac> | restbase deploy start of 2a3972a on canary restbase1008 | [production] | 
            
  | 08:57 | <hoo> | Restarted blazegraph on wdqs1002 (unresponsive) | [production] | 
            
  | 08:12 | <hoo> | Restarted blazegraph on wdqs1002 (unresponsive) | [production] | 
            
  | 05:52 | <gehel> | restarting elasticsearch server elastic1008.eqiad.wmnet (T110236) | [production] | 
            
  | 04:55 | <gehel> | restarting elasticsearch server elastic1007.eqiad.wmnet (T110236) | [production] | 
            
  | 03:27 | <ori> | restarted parsoid on all wtp* hosts to kill any requests from changepropagation service that may have been in flight | [production] | 
            
  | 03:20 | <ori> | ran: sudo salt 'scb*' cmd.run 'puppet agent --disable ; service changeprop stop' | [production] | 
            
  | 03:15 | <ori> | and restarted parsoid on wtp1* | [production] | 
            
  | 03:14 | <ori> | injected an early return to v1Wt2html in routes.js when oldid = 106801025 | [production] | 
            
  | 02:49 | <mwdeploy@tin> | sync-l10n completed (1.27.0-wmf.22) (duration: 18m 23s) | [production] | 
            
  | 00:27 | <twentyafterfour> | phabricator upgrade complete | [production] | 
            
  | 00:21 | <catrope@tin> | Synchronized wmf-config/CommonSettings.php: Plumbing for wmgEchoCrossWikiByDefault (unused for now) (duration: 00m 24s) | [production] | 
            
  | 00:16 | <catrope@tin> | Synchronized wmf-config/InitialiseSettings.php: Plumbing for wmgEchoCrossWikiByDefault (unused for now) (duration: 00m 26s) | [production] | 
            
  | 00:06 | <aude@tin> | rebuilt wikiversions.php and synchronized wikiversions files: Put wikidata on wmf/1.27.0-wmf.23 | [production] | 
            
  | 00:04 | <twentyafterfour> | taking phabricator offline for maintenance | [production] | 
            
  
    | 2016-05-04
      
      § | 
    
  | 23:54 | <aude@tin> | Synchronized php-1.27.0-wmf.23/extensions/Wikidata: Fix bug in other languages box: T134432 (duration: 02m 18s) | [production] | 
            
  | 23:37 | <catrope@tin> | Synchronized wmf-config/InitialiseSettings.php: Remove emailuser override for hewiki, no longer needed (duration: 00m 33s) | [production] | 
            
  | 23:21 | <catrope@tin> | Synchronized php-1.27.0-wmf.23/extensions/VisualEditor: SWAT (duration: 00m 29s) | [production] | 
            
  | 23:18 | <catrope@tin> | Synchronized php-1.27.0-wmf.23/extensions/Cite: SWAT (duration: 00m 52s) | [production] | 
            
  | 21:56 | <demon@tin> | Synchronized wmf-config/CommonSettings.php: (no message) (duration: 00m 28s) | [production] | 
            
  | 21:43 | <cscott> | restarting OCG after puppet deploy of https://gerrit.wikimedia.org/r/286068 | [production] | 
            
  | 21:37 | <ejegg> | updated payments-wiki from c502ab2f6b6ff914d67503a664d36076fdc32dcf to 8962b558b5eb5a4dba91be7ed7649b532fa4ee35 | [production] | 
            
  | 21:35 | <catrope@tin> | Synchronized php-1.27.0-wmf.22/extensions/Echo/Hooks.php: Debug logging for seemingly unattached users (duration: 00m 25s) | [production] | 
            
  | 21:25 | <gehel> | deploying new icinga check on response time for WDQS | [production] | 
            
  | 21:23 | <legoktm@tin> | Synchronized wmf-config/: touch the directory this time (duration: 00m 37s) | [production] |