2001-2050 of 10000 results (29ms)
2021-10-07 §
13:10 <jiji@deploy1002> helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. [production]
13:09 <jiji@deploy1002> helmfile [staging-codfw] START helmfile.d/admin 'apply'. [production]
13:06 <cmjohnson@cumin1001> START - Cookbook sre.dns.netbox [production]
13:06 <volans@cumin2002> END (PASS) - Cookbook sre.experimental.reimage (exit_code=0) for host sretest1001.eqiad.wmnet [production]
13:05 <jiji@deploy1002> helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. [production]
13:05 <jiji@deploy1002> helmfile [staging-codfw] START helmfile.d/admin 'apply'. [production]
12:56 <cmjohnson@cumin1001> START - Cookbook sre.dns.netbox [production]
12:56 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
12:51 <cmjohnson@cumin1001> START - Cookbook sre.dns.netbox [production]
12:40 <volans@cumin2002> START - Cookbook sre.experimental.reimage for host sretest1001.eqiad.wmnet [production]
12:16 <moritzm> installing testvm2005 [production]
11:59 <moritzm> installing openssl security updates for stretch (buster/bullseye already fixed) [production]
11:52 <Lucas_WMDE> EU backport+config window (aka UTC morning) done [production]
11:52 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
11:50 <lucaswerkmeister-wmde@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:725858|Enable Content and Section Translation to Kurdish WP (T290238)]] (duration: 01m 04s) [production]
11:49 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
11:44 <lucaswerkmeister-wmde@deploy1002> Synchronized php-1.38.0-wmf.3/extensions/WikidataPageBanner/includes/WikidataPageBannerFunctions.php: Backport: [[gerrit:727188|Change PropertyId to NumericPropertyId (T289125, T292667)]] (duration: 01m 05s) [production]
11:41 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
11:38 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
11:10 <jbond> update puppet stdlib gerrit:726872 [production]
09:36 <jmm@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
09:31 <jmm@cumin2002> START - Cookbook sre.dns.netbox [production]
09:27 <jmm@cumin2002> END (FAIL) - Cookbook sre.hosts.decommission (exit_code=1) for hosts testvm2004.codfw.wmnet [production]
09:26 <mvernon@cumin2002> END (PASS) - Cookbook sre.experimental.reimage (exit_code=0) for host ms-be2045.codfw.wmnet [production]
09:25 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host testvm2005.codfw.wmnet [production]
09:19 <jmm@cumin2002> START - Cookbook sre.hosts.decommission for hosts testvm2004.codfw.wmnet [production]
09:08 <jmm@cumin2002> START - Cookbook sre.ganeti.makevm for new host testvm2005.codfw.wmnet [production]
08:49 <mvernon@cumin2002> START - Cookbook sre.experimental.reimage for host ms-be2045.codfw.wmnet [production]
08:36 <moritzm> imported jenkins 2.303.2 to thirdparty/ci component for buster-wikimedia [production]
07:57 <Emperor> re-enabling puppet on ms-be2045 after hw work T290881 [production]
07:39 <jayme@deploy1002> helmfile [eqiad] DONE helmfile.d/admin 'sync'. [production]
07:39 <jayme@deploy1002> helmfile [eqiad] START helmfile.d/admin 'sync'. [production]
07:38 <jayme@deploy1002> helmfile [codfw] DONE helmfile.d/admin 'sync'. [production]
07:37 <jayme@deploy1002> helmfile [codfw] START helmfile.d/admin 'sync'. [production]
07:34 <jayme@deploy1002> helmfile [staging-eqiad] DONE helmfile.d/admin 'sync'. [production]
07:33 <jayme@deploy1002> helmfile [staging-eqiad] START helmfile.d/admin 'sync'. [production]
07:32 <jayme@deploy1002> helmfile [staging-codfw] DONE helmfile.d/admin 'sync'. [production]
07:31 <jayme@deploy1002> helmfile [staging-codfw] START helmfile.d/admin 'sync'. [production]
06:21 <ryankemper> [Elastic] Restart of `relforge` complete [production]
06:05 <ryankemper> [Elastic] Cluster in green status, proceeding to next and final node => `ryankemper@relforge1003:~$ sudo systemctl restart elasticsearch_6@relforge-eqiad-small-alpha.service && sudo systemctl restart elasticsearch_6@relforge-eqiad.service` [production]
05:53 <ryankemper> [Elastic] `ryankemper@relforge1004:~$ sudo systemctl restart elasticsearch_6@relforge-eqiad-small-alpha.service && sudo systemctl restart elasticsearch_6@relforge-eqiad.service` [production]
05:48 <ryankemper> [Elastic] Performing rolling restarts of `relforge`. `relforge1003` is the master so I'll restart `relforge1004` first to minimize disruption [production]
03:00 <ejegg> updated payments-wiki from 23d0ffac66 to 6d3560d083 [production]
02:30 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
02:28 <tstarling@deploy1002> Synchronized wmf-config/CommonSettings.php: enable Parsoid API everywhere (duration: 01m 04s) [production]
02:27 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
00:11 <mutante> [grafana2001:~] $ sudo systemctl start rsync-var-lib-grafana because of "PROBLEM - Check systemd state on grafana2001 is CRITICAL: CRITICAL - degraded" because of some race condition where a file vanished during sync [production]
2021-10-06 §
23:57 <mutante> releases2002 - rm /srv/org/wikimedia/reprepro/conf/distributions - contains only jessie-mediawiki - see 725670 and EOL of MediaWiki 1.31 [production]
23:32 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
23:29 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]