| 
      
        2022-01-07
      
      §
     | 
  
    
  | 12:17 | 
  <mwdebug-deploy@deploy1002> | 
  helmfile [eqiad] START helmfile.d/services/mwdebug: apply on pinkunicorn | 
  [production] | 
            
  | 12:14 | 
  <taavi@deploy1002> | 
  Synchronized php-1.38.0-wmf.16/extensions/ProofreadPage/modules/page: Backport: [[gerrit:751843|Makes sure $imgContHorizontal is always initialized (T298694)]] (duration: 00m 59s) | 
  [production] | 
            
  | 11:57 | 
  <mwdebug-deploy@deploy1002> | 
  helmfile [codfw] DONE helmfile.d/services/mwdebug: sync on pinkunicorn | 
  [production] | 
            
  | 11:56 | 
  <mwdebug-deploy@deploy1002> | 
  helmfile [codfw] START helmfile.d/services/mwdebug: apply on pinkunicorn | 
  [production] | 
            
  | 11:56 | 
  <taavi@deploy1002> | 
  Synchronized php-1.38.0-wmf.16/extensions/Flow: Backport: [[gerrit:752014|Revert "Use strict equality when safe to do so" (T298760)]] (duration: 01m 00s) | 
  [production] | 
            
  | 11:56 | 
  <mwdebug-deploy@deploy1002> | 
  helmfile [eqiad] DONE helmfile.d/services/mwdebug: sync on pinkunicorn | 
  [production] | 
            
  | 11:54 | 
  <mwdebug-deploy@deploy1002> | 
  helmfile [eqiad] START helmfile.d/services/mwdebug: apply on pinkunicorn | 
  [production] | 
            
  | 10:40 | 
  <btullis@cumin1001> | 
  END (PASS) - Cookbook sre.aqs.roll-restart (exit_code=0) for AQS aqs cluster: Roll restart of all AQS's nodejs daemons. | 
  [production] | 
            
  | 10:33 | 
  <btullis@cumin1001> | 
  START - Cookbook sre.aqs.roll-restart for AQS aqs cluster: Roll restart of all AQS's nodejs daemons. | 
  [production] | 
            
  | 07:27 | 
  <marostegui@cumin1001> | 
  dbctl commit (dc=all): 'Repooling after maintenance db1098:3316 (T297191)', diff saved to https://phabricator.wikimedia.org/P18413 and previous config saved to /var/cache/conftool/dbconfig/20220107-072742-marostegui.json | 
  [production] | 
            
  | 07:12 | 
  <marostegui@cumin1001> | 
  dbctl commit (dc=all): 'Repooling after maintenance db1098:3316', diff saved to https://phabricator.wikimedia.org/P18412 and previous config saved to /var/cache/conftool/dbconfig/20220107-071237-marostegui.json | 
  [production] | 
            
  | 06:57 | 
  <marostegui@cumin1001> | 
  dbctl commit (dc=all): 'Repooling after maintenance db1098:3316', diff saved to https://phabricator.wikimedia.org/P18411 and previous config saved to /var/cache/conftool/dbconfig/20220107-065733-marostegui.json | 
  [production] | 
            
  | 06:42 | 
  <marostegui@cumin1001> | 
  dbctl commit (dc=all): 'Repooling after maintenance db1098:3316 (T297191)', diff saved to https://phabricator.wikimedia.org/P18410 and previous config saved to /var/cache/conftool/dbconfig/20220107-064228-marostegui.json | 
  [production] | 
            
  | 06:41 | 
  <marostegui@cumin1001> | 
  dbctl commit (dc=all): 'Depooling db1098:3316 (T297191)', diff saved to https://phabricator.wikimedia.org/P18409 and previous config saved to /var/cache/conftool/dbconfig/20220107-064119-marostegui.json | 
  [production] | 
            
  | 06:41 | 
  <marostegui@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1098.eqiad.wmnet with reason: Maintenance | 
  [production] | 
            
  | 06:41 | 
  <marostegui@cumin1001> | 
  START - Cookbook sre.hosts.downtime for 6:00:00 on db1098.eqiad.wmnet with reason: Maintenance | 
  [production] | 
            
  | 06:15 | 
  <marostegui@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2141.codfw.wmnet with reason: Maintenance | 
  [production] | 
            
  | 06:15 | 
  <marostegui@cumin1001> | 
  START - Cookbook sre.hosts.downtime for 6:00:00 on db2141.codfw.wmnet with reason: Maintenance | 
  [production] | 
            
  | 06:15 | 
  <marostegui@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2124.codfw.wmnet with reason: Maintenance | 
  [production] | 
            
  | 06:15 | 
  <marostegui@cumin1001> | 
  START - Cookbook sre.hosts.downtime for 6:00:00 on db2124.codfw.wmnet with reason: Maintenance | 
  [production] | 
            
  | 06:15 | 
  <marostegui@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2117.codfw.wmnet with reason: Maintenance | 
  [production] | 
            
  | 06:15 | 
  <marostegui@cumin1001> | 
  START - Cookbook sre.hosts.downtime for 6:00:00 on db2117.codfw.wmnet with reason: Maintenance | 
  [production] | 
            
  | 06:15 | 
  <marostegui@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2114.codfw.wmnet with reason: Maintenance | 
  [production] | 
            
  | 06:15 | 
  <marostegui@cumin1001> | 
  START - Cookbook sre.hosts.downtime for 6:00:00 on db2114.codfw.wmnet with reason: Maintenance | 
  [production] | 
            
  | 06:15 | 
  <marostegui@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2089.codfw.wmnet with reason: Maintenance | 
  [production] | 
            
  | 06:15 | 
  <marostegui@cumin1001> | 
  START - Cookbook sre.hosts.downtime for 6:00:00 on db2089.codfw.wmnet with reason: Maintenance | 
  [production] | 
            
  | 06:14 | 
  <marostegui@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db[2076,2095].codfw.wmnet with reason: Maintenance | 
  [production] | 
            
  | 06:14 | 
  <marostegui@cumin1001> | 
  START - Cookbook sre.hosts.downtime for 6:00:00 on db[2076,2095].codfw.wmnet with reason: Maintenance | 
  [production] | 
            
  | 05:47 | 
  <marostegui> | 
  rename wikishared.wikimedia_editor_tasks_targets_passed on db1120 T264225 | 
  [production] | 
            
  | 00:23 | 
  <thcipriani@deploy1002> | 
  Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:752036|viwiktionary: add namespaces "Appendix" and "Appendix talk" (T298289)]] (duration: 00m 59s) | 
  [production] | 
            
  | 00:17 | 
  <mwdebug-deploy@deploy1002> | 
  helmfile [codfw] DONE helmfile.d/services/mwdebug: sync on pinkunicorn | 
  [production] | 
            
  | 00:16 | 
  <mwdebug-deploy@deploy1002> | 
  helmfile [codfw] START helmfile.d/services/mwdebug: apply on pinkunicorn | 
  [production] | 
            
  | 00:16 | 
  <mwdebug-deploy@deploy1002> | 
  helmfile [eqiad] DONE helmfile.d/services/mwdebug: sync on pinkunicorn | 
  [production] | 
            
  | 00:15 | 
  <mwdebug-deploy@deploy1002> | 
  helmfile [eqiad] START helmfile.d/services/mwdebug: apply on pinkunicorn | 
  [production] | 
            
  
    | 
      
        2022-01-06
      
      §
     | 
  
    
  | 23:52 | 
  <jhathaway> | 
  bouncing blazegraph on wdqs1004 | 
  [production] | 
            
  | 23:23 | 
  <cmjohnson@cumin1001> | 
  END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host an-test-coord1002.eqiad.wmnet with OS buster | 
  [production] | 
            
  | 22:55 | 
  <cmjohnson@cumin1001> | 
  START - Cookbook sre.hosts.reimage for host an-test-coord1002.eqiad.wmnet with OS buster | 
  [production] | 
            
  | 22:29 | 
  <mwdebug-deploy@deploy1002> | 
  helmfile [codfw] DONE helmfile.d/services/mwdebug: sync on pinkunicorn | 
  [production] | 
            
  | 22:27 | 
  <mwdebug-deploy@deploy1002> | 
  helmfile [codfw] START helmfile.d/services/mwdebug: apply on pinkunicorn | 
  [production] | 
            
  | 22:27 | 
  <mwdebug-deploy@deploy1002> | 
  helmfile [eqiad] DONE helmfile.d/services/mwdebug: sync on pinkunicorn | 
  [production] | 
            
  | 22:26 | 
  <mwdebug-deploy@deploy1002> | 
  helmfile [eqiad] START helmfile.d/services/mwdebug: apply on pinkunicorn | 
  [production] | 
            
  | 22:25 | 
  <twentyafterfour@deploy1002> | 
  rebuilt and synchronized wikiversions files: all wikis to 1.38.0-wmf.16  refs T293958 | 
  [production] | 
            
  | 22:14 | 
  <twentyafterfour@deploy1002> | 
  Synchronized php-1.38.0-wmf.16/extensions/Scribunto/: sync Scribunto to deploy https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Scribunto/+/752006/ (duration: 01m 08s) | 
  [production] | 
            
  | 22:01 | 
  <mwdebug-deploy@deploy1002> | 
  helmfile [codfw] DONE helmfile.d/services/mwdebug: sync on pinkunicorn | 
  [production] | 
            
  | 22:00 | 
  <mwdebug-deploy@deploy1002> | 
  helmfile [codfw] START helmfile.d/services/mwdebug: apply on pinkunicorn | 
  [production] | 
            
  | 22:00 | 
  <mwdebug-deploy@deploy1002> | 
  helmfile [eqiad] DONE helmfile.d/services/mwdebug: sync on pinkunicorn | 
  [production] | 
            
  | 21:59 | 
  <mwdebug-deploy@deploy1002> | 
  helmfile [eqiad] START helmfile.d/services/mwdebug: apply on pinkunicorn | 
  [production] | 
            
  | 20:23 | 
  <ebernhardson@deploy1002> | 
  Finished deploy [wikimedia/discovery/analytics@3297991]: update rdf-spark-tools jar to 0.3.98 (duration: 02m 15s) | 
  [production] | 
            
  | 20:21 | 
  <ebernhardson@deploy1002> | 
  Started deploy [wikimedia/discovery/analytics@3297991]: update rdf-spark-tools jar to 0.3.98 | 
  [production] | 
            
  | 20:19 | 
  <inflatador> | 
  banned elastic2051 from both chi and omega search clusters - T298674 | 
  [production] |