3851-3900 of 10000 results (40ms)
2021-08-11 ยง
20:19 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
19:45 <cmjohnson@cumin1001> START - Cookbook sre.dns.netbox [production]
19:39 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
19:36 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
19:35 <cmjohnson@cumin1001> START - Cookbook sre.dns.netbox [production]
19:30 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
19:29 <jhuneidi@deploy1002> Synchronized php: group1 wikis to 1.37.0-wmf.18 refs T281159 (duration: 01m 08s) [production]
19:28 <jhuneidi@deploy1002> rebuilt and synchronized wikiversions files: group1 wikis to 1.37.0-wmf.18 refs T281159 [production]
19:22 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
19:19 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
19:10 <jhuneidi@deploy1002> rebuilt and synchronized wikiversions files: group0 wikis to 1.37.0-wmf.18 refs T281159 [production]
19:01 <jgleeson> payments-wiki updated from a70aaa7944 to 0a27dbe9b6 [production]
18:32 <hnowlan@cumin1001> END (PASS) - Cookbook sre.cassandra.roll-restart (exit_code=0) for nodes matching A:restbase-eqiad: Restarting to pick up Java security updates - hnowlan@cumin1001 [production]
18:24 <legoktm@deploy1002> helmfile [codfw] DONE helmfile.d/admin 'apply'. [production]
18:23 <legoktm@deploy1002> helmfile [codfw] START helmfile.d/admin 'apply'. [production]
18:23 <legoktm@deploy1002> helmfile [eqiad] DONE helmfile.d/admin 'apply'. [production]
18:22 <legoktm@deploy1002> helmfile [eqiad] START helmfile.d/admin 'apply'. [production]
18:22 <legoktm@deploy1002> helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. [production]
18:21 <bstorm> removed thirdparty/kubeadm-k8s-1-17 in reprepro [production]
18:21 <legoktm@deploy1002> helmfile [staging-eqiad] START helmfile.d/admin 'apply'. [production]
18:20 <legoktm@deploy1002> helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. [production]
18:19 <legoktm@deploy1002> helmfile [staging-codfw] START helmfile.d/admin 'apply'. [production]
18:04 <ebernhardson@deploy1002> Finished deploy [wikimedia/discovery/analytics@563f876]: process_sparql_query: increase parallelism to help backfill (duration: 02m 21s) [production]
18:02 <ebernhardson@deploy1002> Started deploy [wikimedia/discovery/analytics@563f876]: process_sparql_query: increase parallelism to help backfill [production]
17:37 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
17:36 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
17:35 <jforrester@deploy1002> Synchronized php-1.37.0-wmf.18/includes/specials/pagers/ContribsPager.php: T288563 Don't explode Special:Contributions on extension-formatted rows (3/3) (duration: 01m 06s) [production]
17:34 <jforrester@deploy1002> Synchronized php-1.37.0-wmf.18/includes/Revision/RevisionFactory.php: T288563 Don't explode Special:Contributions on extension-formatted rows (2/3) (duration: 01m 08s) [production]
17:32 <jforrester@deploy1002> Synchronized php-1.37.0-wmf.18/includes/Revision/RevisionStore.php: T288563 Don't explode Special:Contributions on extension-formatted rows (1/3) (duration: 01m 09s) [production]
16:22 <dancy> Results of testing php_fpm_always_restart: php_fpm_always_restart=false: 1m19.942s php_fpm_always_restart=true: 3m12.836s [production]
16:19 <dancy@deploy1002> Synchronized README: Testing scap php-rpm rolling restart (after) (duration: 03m 12s) [production]
16:16 <thcipriani> moment of truth for php-fpm-always-restart in scap [production]
16:10 <btullis@cumin1001> END (PASS) - Cookbook sre.druid.roll-restart-workers (exit_code=0) for Druid analytics cluster: Roll restart of Druid jvm daemons. - btullis@cumin1001 [production]
16:05 <dancy@deploy1002> Synchronized README: Testing scap php-rpm rolling restart (before) (duration: 01m 19s) [production]
15:37 <hnowlan@cumin1001> START - Cookbook sre.cassandra.roll-restart for nodes matching A:restbase-eqiad: Restarting to pick up Java security updates - hnowlan@cumin1001 [production]
15:12 <moritzm> import openjdk-8 8u302-b08-1+wmf1 to bullseye-wikimedia (bootstrap build, not to be used yet) T287960 [production]
15:02 <sukhe@cumin1001> END (FAIL) - Cookbook sre.hosts.decommission (exit_code=1) for hosts bast4002.wikimedia.org [production]
14:57 <btullis@cumin1001> START - Cookbook sre.druid.roll-restart-workers for Druid analytics cluster: Roll restart of Druid jvm daemons. - btullis@cumin1001 [production]
14:55 <sukhe@cumin1001> START - Cookbook sre.hosts.decommission for hosts bast4002.wikimedia.org [production]
14:55 <sukhe@cumin1001> END (FAIL) - Cookbook sre.hosts.decommission (exit_code=99) for hosts bast4002.wikimedia.org [production]
14:44 <sukhe@cumin1001> START - Cookbook sre.hosts.decommission for hosts bast4002.wikimedia.org [production]
14:44 <sukhe> s/depool/decommission bast4002.wikimedia.org - T288579 [production]
14:43 <sukhe> depool bast4002.wikimedia.org - T288579 [production]
14:23 <moritzm> installing mx2002 T286911 [production]
14:21 <hnowlan> disabled cassandra-metrics-collector on maps* [production]
13:33 <moritzm> installing Java 8/Java 11 security updates on various analytics hosts [production]
13:29 <btullis@cumin1001> END (PASS) - Cookbook sre.druid.roll-restart-workers (exit_code=0) for Druid analytics cluster: Roll restart of Druid jvm daemons. - btullis@cumin1001 [production]
12:45 <moritzm> imported openjdk-8 8u302-b08-1~deb10u1 to component/jdk8 for buster-wikimedia (forward port of the latest Java 8 security release) [production]
12:32 <godog> roll-restart prometheus T284213 [production]
12:16 <moritzm> installing c-ares security updates on stretch [production]