| 2018-01-04
      
      § | 
    
  | 12:02 | <mobrovac@tin> | Started deploy [mathoid/deploy@c9957ce]: Mathoid v0.7.1 - T172767 | [production] | 
            
  | 12:00 | <moritzm> | upgrading HHVM on API canaries (mw1276-mw1279) to HHVM 3.18.6 | [production] | 
            
  | 10:51 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Repool db1079 - T163190 (duration: 01m 01s) | [production] | 
            
  | 10:39 | <marostegui> | Stop replication in sync on db1079 and db1101:3317 - T163190 | [production] | 
            
  | 10:39 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Depool db1079 - T163190 (duration: 01m 02s) | [production] | 
            
  | 10:16 | <mobrovac@tin> | Finished deploy [mathoid/deploy@7f664ff]: Update Mathoid in codfw to v0.7.0, take #2 - T183557 (duration: 02m 38s) | [production] | 
            
  | 10:14 | <mobrovac@tin> | Started deploy [mathoid/deploy@7f664ff]: Update Mathoid in codfw to v0.7.0, take #2 - T183557 | [production] | 
            
  | 09:58 | <jynus> | restart and upgrade db2053 | [production] | 
            
  | 09:43 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Depool db1101:3317 - T163190 (duration: 03m 09s) | [production] | 
            
  | 09:38 | <moritzm> | rebooting mw1307 and wtp1025 for kernel update | [production] | 
            
  | 09:13 | <moritzm> | rebooting kubernetes1001 for kernel update | [production] | 
            
  | 08:57 | <elukey> | set sysctl -w net.netfilter.nf_conntrack_tcp_timeout_time_wait=65 on mw133[67] (new jobrunners) | [production] | 
            
  | 08:53 | <marostegui> | Fixing inconsistencies on s7 - T163190 | [production] | 
            
  | 08:48 | <marostegui> | Deploy schema change on db1069 (s7) - T174569 | [production] | 
            
  | 08:46 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Repool db1079 - T174569 (duration: 01m 02s) | [production] | 
            
  | 06:48 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Depool db1079 - T174569 (duration: 01m 02s) | [production] | 
            
  | 06:48 | <marostegui> | Deploy schema change on db1079 (s7) with replication enabled - this will generate lag on labs replicas - T174569 | [production] | 
            
  | 06:27 | <marostegui> | Deploy schema change on db1068 (s4) master - T174569 | [production] | 
            
  | 06:23 | <marostegui> | Issue a BBU re-learn cycle on db1059 - T184160 | [production] | 
            
  | 02:49 | <legoktm@tin> | Synchronized php-1.31.0-wmf.15/extensions/Flow/Hooks.php: Fix CheckUser type check thingy - T182834 (duration: 01m 01s) | [production] | 
            
  | 02:25 | <l10nupdate@tin> | scap sync-l10n completed (1.31.0-wmf.12) (duration: 07m 50s) | [production] | 
            
  | 01:50 | <ladsgroup@tin> | Synchronized dblists/group0.dblist: SWAT: Move testwiki2 from group0 to group1 (T182326) (duration: 01m 02s) | [production] | 
            
  
    | 2018-01-03
      
      § | 
    
  | 23:02 | <twentyafterfour> | restarted apache on phab1001 to clear hung workers (refs T182832) | [production] | 
            
  | 22:31 | <bd808@tin> | Finished deploy [striker/deploy@69f1b15]: Enhance membership request workflow and fix Diffusion repo creation (T168027, T182142) (duration: 00m 31s) | [production] | 
            
  | 22:31 | <bd808@tin> | Started deploy [striker/deploy@69f1b15]: Enhance membership request workflow and fix Diffusion repo creation (T168027, T182142) | [production] | 
            
  | 21:41 | <ejegg> | re-enabled ingenico audit | [production] | 
            
  | 21:27 | <twentyafterfour@tin> | Synchronized php: group1 wikis to 1.31.0-wmf.15 (duration: 01m 01s) | [production] | 
            
  | 21:26 | <twentyafterfour@tin> | rebuilt and synchronized wikiversions files: group1 wikis to 1.31.0-wmf.15 | [production] | 
            
  | 21:26 | <twentyafterfour> | deploying 1.31.0-wmf.15 to "Group 1" wikis | [production] | 
            
  | 21:01 | <ottomata> | deleting stale topics from main kafka clusters: T149594 | [production] | 
            
  | 20:56 | <mutante> | uranium - revoked puppet cert, node deactivate, removing from DNS (T183209) | [production] | 
            
  | 20:50 | <mutante> | uranium (ex-ganglia-web) is going into eternal downtime on Icinga.. shutdown -h RIP (T183209) | [production] | 
            
  | 20:23 | <thcipriani> | updateCollation for eswiki running in screen as thcipriani on terbium | [production] | 
            
  | 20:19 | <thcipriani@tin> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:398823|Do not enable lua fine grained tracking for any wiki]] T172914 (duration: 01m 02s) | [production] | 
            
  | 20:16 | <thcipriani@tin> | Synchronized php-1.31.0-wmf.15/extensions/VisualEditor/lib/ve: SWAT: [[gerrit:401771|Update VE core submodule to master]]  T182907 T183590 (duration: 01m 06s) | [production] | 
            
  | 20:06 | <thcipriani@tin> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:396581|Close wikimania2017.wikimedia.org]] PART II T182493 (duration: 01m 04s) | [production] | 
            
  | 20:04 | <thcipriani@tin> | Synchronized dblists/closed.dblist: SWAT: [[gerrit:396581|Close wikimania2017.wikimedia.org]] PART I T182493 (duration: 01m 02s) | [production] | 
            
  | 19:52 | <thcipriani@tin> | Synchronized wmf-config/CommonSettings.php: SWAT: [[gerrit:385953|Extension:Translate default permissions for Wikimedia wikis]] T178793 (duration: 01m 02s) | [production] | 
            
  | 19:42 | <thcipriani@tin> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:401081|Set category collation to uca-es-u-kn for eswiki]] T183802 (duration: 01m 02s) | [production] | 
            
  | 19:22 | <thcipriani@tin> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:399982|Setup some namespace aliases for eswiki]] T183612 (duration: 01m 02s) | [production] | 
            
  | 19:14 | <thcipriani@tin> | Synchronized wmf-config/Wikibase-production.php: SWAT: [[gerrit:401612|Add configuration deboosting scientific articles on Wikidata]] T183510 (duration: 01m 02s) | [production] | 
            
  | 18:53 | <volans> | restarted ircecho on einsteinium | [production] | 
            
  | 18:37 | <moritzm> | uploaded hhvm 3.18.5+dfsg-1+wmf2+deb9u1 for stretch-wikimedia to apt.wikimedia.org | [production] | 
            
  | 18:25 | <ottomata> | deploying change to produce statsv metrics to main kafka clusters from varnishkafka.  statsv on hafnium will be restarted to consume from main.  might cause a short blip in statsv metrics. | [production] | 
            
  | 18:19 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Decom db2028 (duration: 01m 01s) | [production] | 
            
  | 18:17 | <jynus@tin> | Synchronized wmf-config/db-codfw.php: Decom db2028, repool pc2005 (duration: 01m 01s) | [production] | 
            
  | 17:47 | <otto@tin> | Finished deploy [statsv/statsv@362d1a9]: statsv (duration: 00m 02s) | [production] | 
            
  | 17:47 | <otto@tin> | Started deploy [statsv/statsv@362d1a9]: statsv | [production] | 
            
  | 17:35 | <godog> | upload prometheus-jmx-exporter 0.10-3 to jessie/stretch | [production] | 
            
  | 17:35 | <demon@tin> | Synchronized php-1.31.0-wmf.15/extensions/Wikibase: I9da46c36 (duration: 02m 00s) | [production] |