2021-01-06
§
|
16:42 |
<razzi@cumin1001> |
END (PASS) - Cookbook sre.hadoop.roll-restart-masters (exit_code=0) |
[production] |
16:32 |
<jayme@deploy1001> |
helmfile [staging-codfw] DONE helmfile.d/admin 'sync'. |
[production] |
16:31 |
<jayme@deploy1001> |
helmfile [staging-codfw] START helmfile.d/admin 'sync'. |
[production] |
16:15 |
<jayme@deploy1001> |
helmfile [staging-codfw] DONE helmfile.d/admin 'sync'. |
[production] |
16:15 |
<jayme@deploy1001> |
helmfile [staging-codfw] START helmfile.d/admin 'sync'. |
[production] |
16:02 |
<razzi@cumin1001> |
START - Cookbook sre.hadoop.roll-restart-masters |
[production] |
16:01 |
<moritzm> |
installing cups security updates on buster (client-side tools/libs) |
[production] |
15:54 |
<moritzm> |
installing openexr security updates |
[production] |
15:28 |
<jayme> |
imported calico 3.17.1-1 to component/calico-future stretch-wikimedia |
[production] |
15:20 |
<moritzm> |
restarting FPM/Apache on mw canaries to pick up p11-kit update |
[production] |
15:04 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) |
[production] |
15:02 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
15:01 |
<moritzm> |
installing p11-kit security updates on stretch |
[production] |
14:58 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) |
[production] |
14:55 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
13:03 |
<moritzm> |
installing tcpdump security updates |
[production] |
12:11 |
<awight@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:654005|Create rollbacker group on mrwiki (T270864)]] (duration: 01m 21s) |
[production] |
11:33 |
<moritzm> |
installing ruby2.5 security updates |
[production] |
11:18 |
<moritzm> |
installing libjpeg-turbo security updates on buster |
[production] |
11:14 |
<moritzm> |
remove cloudceph2002-dev.wikimedia.org and cloudceph2003-dev.wikimedia.org from debmonitor (got reinstalled as .wmnet) |
[production] |
10:40 |
<jmm@cumin2001> |
dbctl commit (dc=all): 'Depool db2140', diff saved to https://phabricator.wikimedia.org/P13658 and previous config saved to /var/cache/conftool/dbconfig/20210106-104029-jmm.json |
[production] |
10:38 |
<moritzm> |
depooling db2140 T271084 |
[production] |
08:40 |
<moritzm> |
installing Linux 4.9.246 on stretch hosts (no reboots yet) |
[production] |
01:22 |
<mutante> |
testreduce1001 rm -rf /srv/deployment/parsoid/deploy |
[production] |
00:55 |
<eileen> |
process-control config revision is 9bc3d67b02 |
[production] |
00:46 |
<eileen> |
process-control config revision is c38eaa20ed |
[production] |
00:35 |
<eileen> |
civicrm revision changed from 6be8a130df to 1d5f6365ba, config revision is d8756a45c1 |
[production] |
2021-01-05
§
|
23:13 |
<ladsgroup@deploy1001> |
Synchronized php-1.36.0-wmf.25/includes/logging/LogPager.php: [[gerrit:654507|Check for the index name while it's being renamed]] (duration: 01m 06s) |
[production] |
22:26 |
<reedy@deploy1001> |
Synchronized php-1.36.0-wmf.25/extensions/AbuseFilter/extension.json: T271266 (duration: 01m 04s) |
[production] |
21:48 |
<jhuneidi@deploy1001> |
rebuilt and synchronized wikiversions files: Revert "group0 wikis to 1.36.0-wmf.22" |
[production] |
21:12 |
<razzi@cumin1001> |
END (PASS) - Cookbook sre.aqs.roll-restart (exit_code=0) |
[production] |
21:02 |
<razzi@cumin1001> |
START - Cookbook sre.aqs.roll-restart |
[production] |
20:53 |
<razzi@deploy1001> |
Finished deploy [analytics/aqs/deploy@5d05f83]: Configure http request timeout and caching for T268809 (duration: 04m 48s) |
[production] |
20:50 |
<jhuneidi@deploy1001> |
rebuilt and synchronized wikiversions files: group0 wikis to 1.36.0-wmf.25 refs T267418 |
[production] |
20:48 |
<razzi@deploy1001> |
Started deploy [analytics/aqs/deploy@5d05f83]: Configure http request timeout and caching for T268809 |
[production] |
20:44 |
<razzi> |
deploy aqs (analytics query service) as part of analytics train |
[production] |
20:38 |
<rzl> |
rzl@mw1362:~$ sudo -i /usr/local/sbin/restart-php7.2-fpm |
[production] |
20:28 |
<mutante> |
repooled mw1362 |
[production] |
20:20 |
<mutante> |
mw1344 - /usr/local/sbin/restart-php7.2-fpm |
[production] |
20:04 |
<mutante> |
mw1344 - restarted apache2 - it was showing the same "partial results" error a mw1362 - no other appservers are showing up in logstash, but these were #1 and #2 source of errors |
[production] |
19:47 |
<mutante> |
depooled mw1362 |
[production] |
19:41 |
<mutante> |
mw1362 - restarted apache2 |
[production] |
19:29 |
<razzi@deploy1001> |
Finished deploy [analytics/refinery@56fb3ff] (thin): Regular analytics weekly train THIN [analytics/refinery@6ce68c950fc339dc3748cf50e6925cd1031287c4] (duration: 00m 08s) |
[production] |
19:29 |
<razzi@deploy1001> |
Started deploy [analytics/refinery@56fb3ff] (thin): Regular analytics weekly train THIN [analytics/refinery@6ce68c950fc339dc3748cf50e6925cd1031287c4] |
[production] |
19:28 |
<razzi@deploy1001> |
Finished deploy [analytics/refinery@56fb3ff]: Regular analytics weekly train [analytics/refinery@6ce68c950fc339dc3748cf50e6925cd1031287c4] (duration: 09m 37s) |
[production] |
19:19 |
<razzi@deploy1001> |
Started deploy [analytics/refinery@56fb3ff]: Regular analytics weekly train [analytics/refinery@6ce68c950fc339dc3748cf50e6925cd1031287c4] |
[production] |
19:17 |
<razzi> |
deploying refinery for weekly train |
[production] |
19:16 |
<mutante> |
mwdebug1003 - editing apache2 defaults conf and dropping ServerAdmin address.restarting |
[production] |
18:59 |
<jhuneidi@deploy1001> |
Finished scap: testwikis wikis to 1.36.0-wmf.25 refs T267418 (duration: 39m 07s) |
[production] |
18:22 |
<jhuneidi@deploy1001> |
Started scap: testwikis wikis to 1.36.0-wmf.25 refs T267418 |
[production] |