3551-3600 of 10000 results (35ms)
2021-09-20 ยง
13:45 <moritzm> restarting apache on Logstash ELK5 cluster to pick up GNUTLS update T283165 [production]
13:39 <elukey@cumin1001> END (PASS) - Cookbook sre.ores.roll-restart-workers (exit_code=0) for ORES codfw cluster: Roll restart of ORES's daemons. - elukey@cumin1001 [production]
13:20 <elukey@cumin1001> START - Cookbook sre.ores.roll-restart-workers for ORES codfw cluster: Roll restart of ORES's daemons. - elukey@cumin1001 [production]
13:13 <damilare> updated payments-wiki from f9cbf95a12 to 23d0ffac66 [production]
12:59 <jiji@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'tegola-vector-tiles' for release 'main' . [production]
12:58 <marostegui> Drop ct_tag_id_log key from db1144:3314 T277416 [production]
12:54 <moritzm> installing gnutls28 updates for stretch with backport for forthcoming Let's encrypt issuance chain update (T283165) [production]
12:42 <marostegui> Add ct_tag_id_log key to db1144:3314 T277416 [production]
11:50 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
11:48 <hnowlan@cumin1001> START - Cookbook sre.postgresql.postgres-init [production]
11:47 <hnowlan@cumin1001> END (PASS) - Cookbook sre.postgresql.postgres-init (exit_code=0) [production]
11:46 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
11:36 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
11:31 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
11:31 <urbanecm@deploy1002> Finished scap: b9031bc: Mentor dashboard: Mentor tools (T280307) (duration: 11m 44s) [production]
11:23 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
11:20 <urbanecm@deploy1002> Started scap: b9031bc: Mentor dashboard: Mentor tools (T280307) [production]
11:18 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
11:18 <ladsgroup@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:722348|Disable jQuery Migrate on group1 (T280944)]] (duration: 00m 56s) [production]
11:16 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: b518d8ba03e85afdf98f2e06bf569b4f2b551b1b: Mentor dashboard: Enable beta mode at testwiki (T281534) (duration: 00m 55s) [production]
11:11 <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.23/extensions/GrowthExperiments/includes/: b9031bc572f6e3f4e12e6102c2816467af3580f4: Mentor dashboard: Mentor tools (T280307; 5) (duration: 00m 56s) [production]
11:10 <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.23/extensions/GrowthExperiments/ServiceWiring.php: b9031bc572f6e3f4e12e6102c2816467af3580f4: Mentor dashboard: Mentor tools (T280307; 4) (duration: 00m 56s) [production]
11:09 <hnowlan> roll restarting restbase service in codfw [production]
11:08 <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.23/extensions/GrowthExperiments/includes/MentorDashboard/Modules/MentorTools.php: b9031bc572f6e3f4e12e6102c2816467af3580f4: Mentor dashboard: Mentor tools (T280307; 2) (duration: 00m 55s) [production]
11:07 <urbanecm@deploy1002> sync-file aborted: b9031bc572f6e3f4e12e6102c2816467af3580f4: Mentor dashboard: Mentor tools (T280307; 1) (duration: 00m 00s) [production]
11:07 <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.23/extensions/GrowthExperiments/includes/MentorDashboard/MentorTools/MentorStatusManager.php: b9031bc572f6e3f4e12e6102c2816467af3580f4: Mentor dashboard: Mentor tools (T280307; 1) (duration: 00m 57s) [production]
11:05 <hnowlan> roll restarting restbase service in eqiad for openssl updates [production]
10:45 <hnowlan> roll restarting kartotherian and tilerator on maps2* [production]
10:41 <hnowlan> roll restarting kartotherian and tilerator on maps1* [production]
10:36 <jynus> rolling restart bacula & minio daemons on backup hosts [production]
09:59 <moritzm> restarting apache2 on thorium [production]
09:48 <hnowlan@cumin1001> START - Cookbook sre.postgresql.postgres-init [production]
09:47 <marostegui@cumin1001> dbctl commit (dc=all): 'Remove s10 from eqiad T167973', diff saved to https://phabricator.wikimedia.org/P17300 and previous config saved to /var/cache/conftool/dbconfig/20210920-094739-marostegui.json [production]
09:10 <moritzm> installing openssl1.0 updates for stretch with backport for forthcoming Let's encrypt issuance chain update (T283165) [production]
08:35 <moritzm> updating clamav on ticket.wikimedia.org/otrs1001 to 0.103.3 [production]
08:02 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
07:58 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
07:58 <oblivian@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
07:49 <moritzm> uploaded maps-deduped-tilelist 0.0.3~deb10u1 to buster-wikimedia/main T290982 [production]
07:48 <moritzm> uploaded maps-deduped-tilelist 0.0.3~deb10u1 to buster-wikimedia/main [production]
07:48 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
07:43 <oblivian@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
07:43 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
07:35 <marostegui> Stop db1168 and db2129 in sync T167973 [production]
07:34 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
07:34 <urbanecm@deploy1002> Synchronized wmf-config/throttle.php: af9d6e4e29e5f53ad8cf5aa2c235d54500c433bd: Revert "Add throttle rule for Czech wiki course" (duration: 00m 56s) [production]
07:32 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1168 T167973', diff saved to https://phabricator.wikimedia.org/P17299 and previous config saved to /var/cache/conftool/dbconfig/20210920-073256-marostegui.json [production]
07:32 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1096:3316 T167973', diff saved to https://phabricator.wikimedia.org/P17298 and previous config saved to /var/cache/conftool/dbconfig/20210920-073206-marostegui.json [production]
07:31 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1096:3316 T167973', diff saved to https://phabricator.wikimedia.org/P17297 and previous config saved to /var/cache/conftool/dbconfig/20210920-073141-marostegui.json [production]
07:31 <moritzm> uploaded PHP 7.2.34-18+0~20210223.60+debian10~1.gbpb21322+wmf2 to apt.wikimedia.org (component/php7.2 for buster-wikimedia) T291052 [production]