| 2019-10-28
      
      ยง | 
    
  | 23:20 | <catrope@deploy1001> | Synchronized wmf-config/CommonSettings.php: Deploy Echo kask migration to officewiki for testing, part 2 (T222851) (duration: 00m 52s) | [production] | 
            
  | 23:19 | <catrope@deploy1001> | Synchronized wmf-config/ProductionServices.php: Deploy Echo kask migration to officewiki for testing, part 1 (T222851) (duration: 00m 54s) | [production] | 
            
  | 23:18 | <mutante> | re-enabling puppet on moscovium (RT) | [production] | 
            
  | 22:02 | <ejegg> | re-enabled basic fundraising jobs (Queue consumers, audit processors, TY mailer) | [production] | 
            
  | 20:56 | <cdanis> | restart memcached on mc1022 T235188 | [production] | 
            
  | 20:37 | <Jeff_Green> | authdns update to switch fundraising db service hostname | [production] | 
            
  | 20:19 | <ejegg> | disabled all fundraising scheduled jobs | [production] | 
            
  | 19:50 | <rlazarus> | restarted memcached on mc1021 (T235188) | [production] | 
            
  | 19:41 | <ssastry@deploy1001> | Finished deploy [parsoid/deploy@d932d6a]: Update parsoid to 089bf28d (duration: 02m 42s) | [production] | 
            
  | 19:38 | <ssastry@deploy1001> | Started deploy [parsoid/deploy@d932d6a]: Update parsoid to 089bf28d | [production] | 
            
  | 18:53 | <moritzm> | updating PHP on people1001 | [production] | 
            
  | 18:52 | <Urbanecm> | Morning SWAT done | [production] | 
            
  | 18:42 | <urbanecm@deploy1001> | Synchronized wmf-config/logging.php: SWAT: 1a09e2a: Direct Parsoid/PHP logs to a parsoid-php log "type" (T235899) (duration: 00m 52s) | [production] | 
            
  | 18:41 | <rlazarus> | restarted memcached on mc1020 T235188 | [production] | 
            
  | 18:32 | <mutante> | moscovium - rename all files in /etc/request-tracker4/RT_SiteConfig.d to have a .pm extension - this fixed RT - login works again - puppet patch coming up (T180641) | [production] | 
            
  | 18:30 | <urbanecm@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: 30111f3: Enable mapframe at kawiki (T229726) (duration: 00m 53s) | [production] | 
            
  | 18:28 | <mutante> | moscovium - deleting /etc/request-tracker4/RT_SiteConfig.d/ 50-debconf.pm and 51-dbconfig-common.pm which duplicate the same files without .pm extension with wrong values, probably due to some package change (T180641) | [production] | 
            
  | 18:27 | <jgleeson> | updated paymentswiki from 7bb9f5257e to e28bc54e85 | [production] | 
            
  | 18:26 | <urbanecm@deploy1001> | Synchronized wmf-config/: SWAT: c48271d: Revert "Config changes for Echo kask migration" (T222851) (duration: 00m 53s) | [production] | 
            
  | 18:24 | <urbanecm@deploy1001> | Synchronized php-1.35.0-wmf.3/extensions/VisualEditor/includes/ApiVisualEditor.php: SWAT: b19ad5f: Revert "Revert "ApiVisualEditor: Return etag with content for preloaded content""; 4f3b724: ApiVisualEditor: Fix preload handling further (T233320) (duration: 00m 53s) | [production] | 
            
  | 18:15 | <Urbanecm> | Run mwscript namespaceDupes.php --wiki=thwikisource --fix (T236640) | [production] | 
            
  | 18:14 | <urbanecm@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: ea927dd: Rename author NS at thwikisource (T236640) (duration: 00m 53s) | [production] | 
            
  | 18:07 | <urbanecm@deploy1001> | Synchronized wmf-config/: SWAT: ddaa534: Config changes for Echo kask migration (T222851) (duration: 00m 55s) | [production] | 
            
  | 17:20 | <elukey@cumin1001> | END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) | [production] | 
            
  | 17:12 | <bblack> | mr1-eqiad: fix bast3004 access for eqiad mgmt network - T236686 | [production] | 
            
  | 17:11 | <_joe_> | starting rolling restart of memcached servers in eqiad, beginning with mc1019 T235188 | [production] | 
            
  | 17:11 | <bblack> | mr1-codfw: fix bast3004 access for codfw mgmt network - T236686 | [production] | 
            
  | 17:10 | <bblack> | mr1-ulsfo: fix bast3004 access for ulsfo mgmt network - T236686 | [production] | 
            
  | 16:57 | <bblack> | mr1-eqsin: fix bast3004 access for eqsin mgmt network - T236686 | [production] | 
            
  | 16:56 | <elukey@cumin1001> | START - Cookbook sre.ganeti.makevm | [production] | 
            
  | 16:54 | <bblack> | mr1-esams: fix bast3004 access for esams mgmt network - T236686 | [production] | 
            
  | 16:36 | <jbond42> | restart puppetdb on pupetdb1001 to remove queue | [production] | 
            
  | 13:50 | <ema> | pool cp5007 with ATS backend T227432 | [production] | 
            
  | 13:30 | <godog> | roll restart logstash in codfw/eqiad to apply new config | [production] | 
            
  | 13:23 | <effie> | enable puppet on mw1*, depool and repool to reload apache - T229792 | [production] | 
            
  | 13:13 | <effie> | enable puppet on mw[1261-1265].eqiad.wmnet (mw canaries), depool and repool to reload apache - T229792 | [production] | 
            
  | 13:07 | <ema@cumin1001> | END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) | [production] | 
            
  | 13:07 | <ema@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 13:05 | <effie> | enable puppet on mw2* servers, depool and repool to reload apache - T229792 | [production] | 
            
  | 13:01 | <jynus> | stop db1114 for testing | [production] | 
            
  | 12:30 | <ema> | depool cp5007 and reimage as text_ats T227432 | [production] | 
            
  | 12:22 | <effie> | depool mw2150 | [production] | 
            
  | 11:56 | <twentyafterfour@deploy1001> | Finished deploy [phabricator/deployment@e4e2b22]: testing deployment of phabricator to phab1001 (duration: 00m 05s) | [production] | 
            
  | 11:56 | <twentyafterfour@deploy1001> | Started deploy [phabricator/deployment@e4e2b22]: testing deployment of phabricator to phab1001 | [production] | 
            
  | 11:34 | <Urbanecm> | EU SWAT done | [production] | 
            
  | 11:33 | <urbanecm@deploy1001> | Synchronized php-1.35.0-wmf.3/extensions/VisualEditor/includes/ApiVisualEditorEdit.php: SWAT: 8caf681: Dont log missing ETags when creating a new page, thats normal (T233320) (duration: 00m 54s) | [production] | 
            
  | 11:33 | <effie> | Disable puppet on mw* for 545652 - T229792 | [production] | 
            
  | 11:12 | <urbanecm@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: dd2f06c: Add Translate channel for the Translate extension (T221119) (duration: 00m 53s) | [production] | 
            
  | 11:07 | <urbanecm@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: ff17666: Adjust wgUploadNavigationUrl for azwiki to point to commons UpWiz (T236307) (duration: 00m 53s) | [production] | 
            
  | 11:05 | <urbanecm@deploy1001> | Synchronized dblists/commonsuploads.dblist: SWAT: 7e26ef4: Revert "Restrict uploads on azwiki" (T236307) (duration: 00m 53s) | [production] |