| 2019-04-02
      
      ยง | 
    
  | 16:47 | <XioNoX> | - replacing accepted-prefix-limit with prefix-limit in eqsin - T211730 | [production] | 
            
  | 16:44 | <ppchelko@deploy1001> | Finished deploy [restbase/deploy@6026ad1]: Switch to swagger 3 T218218 (duration: 04m 52s) | [production] | 
            
  | 16:39 | <ppchelko@deploy1001> | Started deploy [restbase/deploy@6026ad1]: Switch to swagger 3 T218218 | [production] | 
            
  | 16:36 | <XioNoX> | - replacing accepted-prefix-limit with prefix-limit on esams - T211730 | [production] | 
            
  | 16:12 | <XioNoX> | - replacing accepted-prefix-limit with prefix-limit on cr2-eqiad - T211730 | [production] | 
            
  | 16:02 | <mutante> | T194174 - bump. started alerting again 2 days ago | [production] | 
            
  | 16:00 | <mutante> | icinga - schedule (30d) downtime for kubernetes operational latencies alerts (T219696) on kubernetes1004 | [production] | 
            
  | 15:57 | <arturo> | T219626 reimaging cloudcontrol2001-dev again | [production] | 
            
  | 15:55 | <mutante> | scandium - systemctl start parsoid-vd  was failed (T201366) | [production] | 
            
  | 15:55 | <herron> | beginning rolling upgrade of codfw ELK cluster to 5.6.15 T219571 | [production] | 
            
  | 15:52 | <mutante> | icinga - re-enabling notifications for scandium. setup task is resolved yet systemd is alerting, should not have been turned off anymore (T201366) | [production] | 
            
  | 15:39 | <XioNoX> | repool eqsin - T219847 | [production] | 
            
  | 15:32 | <jbond42> | add cpp-hocon 0.1.6 to jessie-wikimedia/backports | [production] | 
            
  | 15:21 | <jforrester@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: VE: Enable mobile section editing A/B test on all remaining wikis T219564 (duration: 00m 51s) | [production] | 
            
  | 15:07 | <moritzm> | stopped/disabled ipmievd on cumin2001 | [production] | 
            
  | 14:54 | <jbond42> | add leatherman 1.4 to jessie-wikimedia/backports | [production] | 
            
  | 13:44 | <anomie@mwmaint1002> | Fixing empty values for 'target_author_actor' in log_search on test wikis and mediawikiwiki for T215525 | [production] | 
            
  | 13:24 | <volans> | reboot ms-be2026 to see if that fixes the controller - T219854 | [production] | 
            
  | 13:23 | <volans@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 13:23 | <volans@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 13:20 | <volans@cumin1001> | END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) | [production] | 
            
  | 13:20 | <volans@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 13:20 | <jynus> | updating puppet compiler facts | [production] | 
            
  | 12:11 | <arturo> | icinga downtime toolschecker for 1 month T219243 | [production] | 
            
  | 12:07 | <hashar> | contint1001: compressing some MediaWiki debugging logs under /srv/jenkins/builds # T219850 | [production] | 
            
  | 11:42 | <moritzm> | restarting parsoid on wtp1025 to pick up openssl update | [production] | 
            
  | 11:33 | <hashar> | contint1001: cleaning Docker containers #T219850 | [production] | 
            
  | 11:23 | <Amir1> | EU SWAT is done | [production] | 
            
  | 11:22 | <ladsgroup@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:499777|Add the urlshortener-manage-url right and enable it for stewards (T133109)]], Part I (duration: 00m 51s) | [production] | 
            
  | 11:21 | <ladsgroup@deploy1001> | Synchronized wmf-config/CommonSettings.php: SWAT: [[gerrit:499777|Add the urlshortener-manage-url right and enable it for stewards (T133109)]], Part I (duration: 00m 53s) | [production] | 
            
  | 11:14 | <akosiaris> | T217715 Update mathoid, citoid, cxserver, eventgate grafana dashboards to use the new recording rules for the quantiles | [production] | 
            
  | 11:14 | <jbond42> | add cmake 3.6.2 to jessie-wikimedia/backports | [production] | 
            
  | 11:02 | <jbond42> | add rapidjson 1.1.0 to jessie-wikimedia/backports | [production] | 
            
  | 10:47 | <jbond42> | add catch 1.10 to jessie-wikimedia/backports | [production] | 
            
  | 10:42 | <jbond42> | add strip-nondeterminism 0.034 to jessie-wikimedia/backports | [production] | 
            
  | 10:39 | <jbond42> | add dh-autoreconf 12 to jessie-wikimedia/backports | [production] | 
            
  | 10:30 | <jbond42> | add debhelper 10.2.5 and dh-systemd 10.2.5 to jessie-wikimedia/backports | [production] | 
            
  | 10:08 | <elukey> | manually purge varnishkafka graphite alert's URL as attempt to avoid a flapping alert - T219842 | [production] | 
            
  | 09:14 | <arturo> | T219776 finally reimaging cloudnet2003-dev.codfw.wmnet (was labtestnet2003) | [production] | 
            
  | 09:03 | <_joe_> | uploaded patched version of bootstrap-vz to account for jessie-updates vanishing (T219683) | [production] | 
            
  | 08:58 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Repool all slaves in x1 T219777 T143763 (duration: 00m 53s) | [production] | 
            
  | 08:50 | <marostegui> | Execute schema change on db1069 x1 master with replication enabled on the following small wikis: aawiki aawikibooks aawiktionary abwiki abwiktionary acewiki advisorswiki advisorywiki adywiki afwiki T143763 | [production] | 
            
  | 08:20 | <marostegui> | Compress wikishared.urlshortcodes table on x1, directly on the master with replication (table has 1 row) - T219777 | [production] | 
            
  | 08:16 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Depool all slaves in x1 T219777 T143763 (duration: 00m 53s) | [production] | 
            
  | 08:13 | <moritzm> | installing debdeploy updates on remaining hosts in eqiad/codfw | [production] | 
            
  | 08:05 | <moritzm> | installing openssl1.0 security updates | [production] | 
            
  | 07:52 | <moritzm> | removed labvirt1008 from debmonitor (T216661) | [production] | 
            
  | 06:38 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Repool db1120 (duration: 00m 50s) | [production] | 
            
  | 06:31 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Depool db1120 (duration: 00m 52s) | [production] | 
            
  | 06:26 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Repool db1064 (duration: 00m 52s) | [production] |