| 2021-06-10
      
      ยง | 
    
  | 18:39 | <urbanecm@deploy1002> | update-interwiki-cache aborted: Update interwiki cache (duration: 00m 03s) | [production] | 
            
  | 18:38 | <urbanecm@deploy1002> | Synchronized php-1.37.0-wmf.9/extensions/UniversalLanguageSelector/resources/js/ext.uls.launch.js: 8aeab139879613782548b20fc11af5e66589e30a: Fire language change hook (T280770) (duration: 01m 07s) | [production] | 
            
  | 18:05 | <urbanecm@deploy1002> | Synchronized wmf-config/CommonSettings.php: d26968c1c3b3f3e115ff37a9a138d225cabba25a: wgWelcomeSurveyExperimentalGroups: Use new syntax in CS.php (T284597; T284735) (duration: 01m 08s) | [production] | 
            
  | 17:11 | <moritzm> | updating bullseye installer image to latest daily image (kernel ABI changed again) T275873 | [production] | 
            
  | 17:09 | <jgiannelos@deploy1002> | helmfile [eqiad] Ran 'sync' command on namespace 'mobileapps' for release 'production' . | [production] | 
            
  | 17:06 | <jgiannelos@deploy1002> | helmfile [codfw] Ran 'sync' command on namespace 'mobileapps' for release 'production' . | [production] | 
            
  | 17:03 | <jgiannelos@deploy1002> | helmfile [staging] Ran 'sync' command on namespace 'mobileapps' for release 'staging' . | [production] | 
            
  | 16:53 | <razzi@cumin1001> | END (FAIL) - Cookbook sre.hadoop.roll-restart-masters (exit_code=99) | [production] | 
            
  | 16:51 | <moritzm> | installing rails security updates | [production] | 
            
  | 16:37 | <krinkle@deploy1002> | Synchronized wmf-config/CommonSettings.php: no-op for Beta I2a42c222003 (duration: 01m 07s) | [production] | 
            
  | 16:34 | <pt1979@cumin2002> | END (PASS) - Cookbook sre.dns.netbox (exit_code=0) | [production] | 
            
  | 16:29 | <pt1979@cumin2002> | START - Cookbook sre.dns.netbox | [production] | 
            
  | 16:24 | <razzi@cumin1001> | START - Cookbook sre.hadoop.roll-restart-masters | [production] | 
            
  | 15:09 | <papaul> | power down ms-be2038 for BBU replacement | [production] | 
            
  | 12:32 | <marostegui@cumin1001> | dbctl commit (dc=all): 'db1098:3316 (re)pooling @ 100%: Repool db1098:3316', diff saved to https://phabricator.wikimedia.org/P16417 and previous config saved to /var/cache/conftool/dbconfig/20210610-123201-root.json | [production] | 
            
  | 12:16 | <marostegui@cumin1001> | dbctl commit (dc=all): 'db1098:3316 (re)pooling @ 75%: Repool db1098:3316', diff saved to https://phabricator.wikimedia.org/P16416 and previous config saved to /var/cache/conftool/dbconfig/20210610-121657-root.json | [production] | 
            
  | 12:01 | <marostegui@cumin1001> | dbctl commit (dc=all): 'db1098:3316 (re)pooling @ 60%: Repool db1098:3316', diff saved to https://phabricator.wikimedia.org/P16415 and previous config saved to /var/cache/conftool/dbconfig/20210610-120153-root.json | [production] | 
            
  | 11:46 | <marostegui@cumin1001> | dbctl commit (dc=all): 'db1098:3316 (re)pooling @ 50%: Repool db1098:3316', diff saved to https://phabricator.wikimedia.org/P16414 and previous config saved to /var/cache/conftool/dbconfig/20210610-114650-root.json | [production] | 
            
  | 11:31 | <marostegui@cumin1001> | dbctl commit (dc=all): 'db1098:3316 (re)pooling @ 40%: Repool db1098:3316', diff saved to https://phabricator.wikimedia.org/P16413 and previous config saved to /var/cache/conftool/dbconfig/20210610-113146-root.json | [production] | 
            
  | 11:16 | <marostegui@cumin1001> | dbctl commit (dc=all): 'db1098:3316 (re)pooling @ 30%: Repool db1098:3316', diff saved to https://phabricator.wikimedia.org/P16412 and previous config saved to /var/cache/conftool/dbconfig/20210610-111643-root.json | [production] | 
            
  | 11:01 | <marostegui@cumin1001> | dbctl commit (dc=all): 'db1098:3316 (re)pooling @ 20%: Repool db1098:3316', diff saved to https://phabricator.wikimedia.org/P16411 and previous config saved to /var/cache/conftool/dbconfig/20210610-110139-root.json | [production] | 
            
  | 11:00 | <jbond@deploy1002> | Finished deploy [netbox/deploy@e9f2382]: deploy v2.10.4-wmf4 to netbox-next (duration: 00m 53s) | [production] | 
            
  | 10:59 | <jbond@deploy1002> | Started deploy [netbox/deploy@e9f2382]: deploy v2.10.4-wmf4 to netbox-next | [production] | 
            
  | 10:47 | <topranks> | T283163: Adding "metric-out minimum-igp" to BGP group Confed_eqord on eqiad, codfw and eqdfw CRs. | [production] | 
            
  | 10:46 | <marostegui@cumin1001> | dbctl commit (dc=all): 'db1098:3316 (re)pooling @ 10%: Repool db1098:3316', diff saved to https://phabricator.wikimedia.org/P16410 and previous config saved to /var/cache/conftool/dbconfig/20210610-104635-root.json | [production] | 
            
  | 10:43 | <urbanecm@deploy1002> | Synchronized php-1.37.0-wmf.9/extensions/WikiEditor/modules/jquery.wikiEditor.js: 8a17c43c5470b84ba58239bb2cf947dbebf1979f: Fix call to renamed var (T284716) (duration: 01m 25s) | [production] | 
            
  | 10:31 | <marostegui@cumin1001> | dbctl commit (dc=all): 'db1098:3316 (re)pooling @ 5%: Repool db1098:3316', diff saved to https://phabricator.wikimedia.org/P16409 and previous config saved to /var/cache/conftool/dbconfig/20210610-103132-root.json | [production] | 
            
  | 10:30 | <marostegui@cumin1001> | dbctl commit (dc=all): 'Repool db1113:3316', diff saved to https://phabricator.wikimedia.org/P16408 and previous config saved to /var/cache/conftool/dbconfig/20210610-103032-marostegui.json | [production] | 
            
  | 10:29 | <mvolz@deploy1002> | helmfile [eqiad] Ran 'sync' command on namespace 'citoid' for release 'production' . | [production] | 
            
  | 10:28 | <kormat> | running optimize tables against pc1009 (pc3) T282761 | [production] | 
            
  | 10:25 | <mvolz@deploy1002> | helmfile [codfw] Ran 'sync' command on namespace 'citoid' for release 'production' . | [production] | 
            
  | 10:21 | <mvolz@deploy1002> | helmfile [staging] Ran 'sync' command on namespace 'citoid' for release 'staging' . | [production] | 
            
  | 10:18 | <marostegui@cumin1001> | dbctl commit (dc=all): 'db1098:3317 (re)pooling @ 100%: Repool db1098:3317 after schema change', diff saved to https://phabricator.wikimedia.org/P16407 and previous config saved to /var/cache/conftool/dbconfig/20210610-101858-root.json | [production] | 
            
  | 10:03 | <marostegui@cumin1001> | dbctl commit (dc=all): 'db1098:3317 (re)pooling @ 75%: Repool db1098:3317 after schema change', diff saved to https://phabricator.wikimedia.org/P16406 and previous config saved to /var/cache/conftool/dbconfig/20210610-100355-root.json | [production] | 
            
  | 09:48 | <marostegui@cumin1001> | dbctl commit (dc=all): 'db1098:3317 (re)pooling @ 60%: Repool db1098:3317 after schema change', diff saved to https://phabricator.wikimedia.org/P16405 and previous config saved to /var/cache/conftool/dbconfig/20210610-094851-root.json | [production] | 
            
  | 09:33 | <marostegui@cumin1001> | dbctl commit (dc=all): 'db1098:3317 (re)pooling @ 50%: Repool db1098:3317 after schema change', diff saved to https://phabricator.wikimedia.org/P16404 and previous config saved to /var/cache/conftool/dbconfig/20210610-093346-root.json | [production] | 
            
  | 09:30 | <marostegui@cumin1001> | dbctl commit (dc=all): 'Depool db1113:3316', diff saved to https://phabricator.wikimedia.org/P16402 and previous config saved to /var/cache/conftool/dbconfig/20210610-093003-marostegui.json | [production] | 
            
  | 09:22 | <marostegui@cumin1001> | dbctl commit (dc=all): 'Depool db1098:3316', diff saved to https://phabricator.wikimedia.org/P16401 and previous config saved to /var/cache/conftool/dbconfig/20210610-092246-marostegui.json | [production] | 
            
  | 09:18 | <marostegui@cumin1001> | dbctl commit (dc=all): 'db1098:3317 (re)pooling @ 40%: Repool db1098:3317 after schema change', diff saved to https://phabricator.wikimedia.org/P16399 and previous config saved to /var/cache/conftool/dbconfig/20210610-091842-root.json | [production] | 
            
  | 09:03 | <marostegui@cumin1001> | dbctl commit (dc=all): 'db1098:3316 (re)pooling @ 30%: Repool db1098:3316 after schema change', diff saved to https://phabricator.wikimedia.org/P16398 and previous config saved to /var/cache/conftool/dbconfig/20210610-090345-root.json | [production] | 
            
  | 09:03 | <marostegui@cumin1001> | dbctl commit (dc=all): 'db1098:3317 (re)pooling @ 30%: Repool db1098:3317 after schema change', diff saved to https://phabricator.wikimedia.org/P16397 and previous config saved to /var/cache/conftool/dbconfig/20210610-090339-root.json | [production] | 
            
  | 08:48 | <marostegui@cumin1001> | dbctl commit (dc=all): 'db1098:3316 (re)pooling @ 20%: Repool db1098:3316 after schema change', diff saved to https://phabricator.wikimedia.org/P16396 and previous config saved to /var/cache/conftool/dbconfig/20210610-084841-root.json | [production] | 
            
  | 08:48 | <marostegui@cumin1001> | dbctl commit (dc=all): 'db1098:3317 (re)pooling @ 20%: Repool db1098:3317 after schema change', diff saved to https://phabricator.wikimedia.org/P16395 and previous config saved to /var/cache/conftool/dbconfig/20210610-084835-root.json | [production] | 
            
  | 08:33 | <marostegui@cumin1001> | dbctl commit (dc=all): 'db1098:3316 (re)pooling @ 10%: Repool db1098:3316 after schema change', diff saved to https://phabricator.wikimedia.org/P16394 and previous config saved to /var/cache/conftool/dbconfig/20210610-083338-root.json | [production] | 
            
  | 08:33 | <marostegui@cumin1001> | dbctl commit (dc=all): 'db1098:3317 (re)pooling @ 10%: Repool db1098:3317 after schema change', diff saved to https://phabricator.wikimedia.org/P16393 and previous config saved to /var/cache/conftool/dbconfig/20210610-083332-root.json | [production] | 
            
  | 08:25 | <volans> | uploaded spicerack_0.0.53 to apt.wikimedia.org buster-wikimedia,bullseye-wikimedia | [production] | 
            
  | 08:18 | <marostegui@cumin1001> | dbctl commit (dc=all): 'db1098:3316 (re)pooling @ 5%: Repool db1098:3316 after schema change', diff saved to https://phabricator.wikimedia.org/P16392 and previous config saved to /var/cache/conftool/dbconfig/20210610-081834-root.json | [production] | 
            
  | 08:18 | <marostegui@cumin1001> | dbctl commit (dc=all): 'db1098:3317 (re)pooling @ 5%: Repool db1098:3317 after schema change', diff saved to https://phabricator.wikimedia.org/P16391 and previous config saved to /var/cache/conftool/dbconfig/20210610-081828-root.json | [production] | 
            
  | 08:17 | <marostegui> | Drop several grants from labswiki (wikitech) T282074 | [production] | 
            
  | 07:57 | <jynus> | reset-failed on cumin1001 after backup rerun | [production] |