51-100 of 10000 results (21ms)
2020-08-25 §
11:59 <jayme@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'push-notifications' for release 'main' . [production]
11:49 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1118 MCR changes', diff saved to https://phabricator.wikimedia.org/P12339 and previous config saved to /var/cache/conftool/dbconfig/20200825-114938-marostegui.json [production]
11:37 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1118 MCR changes', diff saved to https://phabricator.wikimedia.org/P12338 and previous config saved to /var/cache/conftool/dbconfig/20200825-113758-marostegui.json [production]
11:36 <hnowlan@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'api-gateway' for release 'production' . [production]
11:32 <hnowlan@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'api-gateway' for release 'production' . [production]
11:29 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1118 MCR changes', diff saved to https://phabricator.wikimedia.org/P12337 and previous config saved to /var/cache/conftool/dbconfig/20200825-112859-marostegui.json [production]
11:25 <marostegui> Upgrade mysql on db1118 after MCR change [production]
11:16 <Urbanecm> EU B&C done [production]
11:07 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: d869e308492ee72cb3d1998b15409aa44a4af9c7: Enable ContentTranslation as a default tool in Assamese and Burmese WPs (T258503; T258505) (duration: 01m 00s) [production]
10:59 <moritzm> installing remaining libx11 security updates [production]
10:37 <arturo> import all binary packages from tesseract-ocr-lang into stretch-wikimedia/component/tesseract-410-bpo (T247422) [production]
10:32 <hnowlan@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'api-gateway' for release 'production' . [production]
10:28 <oblivian@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'termbox' for release 'staging' . [production]
10:28 <oblivian@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'termbox' for release 'test' . [production]
10:23 <hnowlan@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'api-gateway' for release 'staging' . [production]
10:23 <moritzm> removed fermium.wikimedia.org from debmonitor [production]
09:45 <marostegui> Create missing table cx_notification_log on x1 wikishared T261190 [production]
08:50 <XioNoX> re-activate eqord peering/transit - T259593 [production]
08:19 <XioNoX> reconfigure eqord to be AS65020 - T259593 [production]
08:18 <XioNoX> deactivate eqord peering/transit - T259593 [production]
07:22 <ayounsi@cumin1001> END (FAIL) - Cookbook sre.network.prepare-upgrade (exit_code=1) [production]
07:13 <marostegui> Upgrade MySQL on dbstore1004 [production]
07:09 <dcausse> depooling wdqs1005 (high lag) [production]
07:04 <dcausse> restartint blazegraph on wdqs1005 (T242453) [production]
06:20 <ayounsi@cumin1001> START - Cookbook sre.network.prepare-upgrade [production]
05:38 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1111, db1118 for MCR change', diff saved to https://phabricator.wikimedia.org/P12336 and previous config saved to /var/cache/conftool/dbconfig/20200825-053856-marostegui.json [production]
05:38 <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool db1084,db1092 after MCR changes', diff saved to https://phabricator.wikimedia.org/P12335 and previous config saved to /var/cache/conftool/dbconfig/20200825-053801-marostegui.json [production]
05:26 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1084,db1092 after MCR changes', diff saved to https://phabricator.wikimedia.org/P12334 and previous config saved to /var/cache/conftool/dbconfig/20200825-052602-marostegui.json [production]
05:21 <moritzm> installing Java security updates on relforge* [production]
05:13 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1084,db1092 after MCR changes', diff saved to https://phabricator.wikimedia.org/P12333 and previous config saved to /var/cache/conftool/dbconfig/20200825-051327-marostegui.json [production]
05:11 <marostegui> Remove revisions triggers from db2094:3311 T238966 [production]
05:10 <marostegui> Deploy MCR schema change on s1 codfw, this will create lag on s1 codfw - T238966 [production]
05:04 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1084,db1092 after MCR changes', diff saved to https://phabricator.wikimedia.org/P12332 and previous config saved to /var/cache/conftool/dbconfig/20200825-050451-marostegui.json [production]
04:02 <ejegg> updated fundraising python tools from 305f2a4438 to dcad0bfe75 [production]
01:49 <eileen> civicrm revision changed from ce28723709 to 0f195c6cca, config revision is 96839009f1 [production]
01:39 <eileen> civicrm revision is ce28723709, config revision is 96839009f1 [production]
01:30 <eileen> civicrm revision is ce28723709, config revision is 54c8c7abf2 [production]
01:17 <cdanis> repool esams [production]
01:11 <cdanis> T259621 wrong junos version was staged on cr2-esams, abandoning this attempt and putting back in service [production]
01:07 <cdanis> cdanis@re0.cr2-esams> request system software add validate re1 /var/tmp/junos-vmhost-install-mx-x86-64-17.3R3-S8.1.tgz [production]
00:56 <cdanis> T259621 ❌cdanis@cumin1001.eqiad.wmnet ~ 🕘🍺 homer 'cr*' commit 'drain cr2-esams transport link' [production]
00:36 <cdanis> T259621 cdanis@re1.cr3-esams> request chassis routing-engine master switch [production]
00:30 <cdanis> T259621 cdanis@re1.cr3-esams> request vmhost reboot re0 [production]
00:24 <cdanis> T259621 cdanis@re1.cr3-esams> request vmhost software add /var/tmp/junos-vmhost-install-mx-x86-64-17.3R3-S8.1.tgz re0 [production]
00:18 <cdanis> T259621 cdanis@re0.cr3-esams> request chassis routing-engine master switch [production]
00:14 <cdanis> T259621 cdanis@re0.cr3-esams> request vmhost reboot re1 [production]
00:08 <cdanis> T259621 cdanis@re0.cr3-esams> request vmhost software add /var/tmp/junos-vmhost-install-mx-x86-64-17.3R3-S8.1.tgz re1 [production]
2020-08-24 §
23:46 <cdanis> depool esams T259621 [production]
23:16 <Urbanecm> Evening B&C window done [production]
23:06 <urbanecm@deploy1001> Synchronized wmf-config/CommonSettings.php: 778f710bbbdb24730f7ce4c75d5ff1ca7a5ce3b3: Alternate configuration mechanism for Parsoid (T241961) (duration: 00m 58s) [production]