|
2020-08-19
ยง
|
| 22:41 |
<eileen> |
civicrm revision changed from 34f95a3311 to c442a09153, config revision is 3cdffd4fc2 |
[production] |
| 21:27 |
<eileen> |
civicrm revision changed from 154519cc1f to 34f95a3311, config revision is 3cdffd4fc2 |
[production] |
| 21:17 |
<cdanis@cumin1001> |
END (PASS) - Cookbook sre.network.cf (exit_code=0) |
[production] |
| 21:17 |
<cdanis@cumin1001> |
START - Cookbook sre.network.cf |
[production] |
| 20:39 |
<dpifke@deploy1001> |
Finished deploy [performance/arc-lamp@2ef1af7]: Deploy fixes for notifications and OOM prevention (T259167) (duration: 00m 06s) |
[production] |
| 20:39 |
<dpifke@deploy1001> |
Started deploy [performance/arc-lamp@2ef1af7]: Deploy fixes for notifications and OOM prevention (T259167) |
[production] |
| 19:43 |
<ebernhardson> |
restart mjolnir-kafka-bulk-daemon on search-loader2001 with debug logging |
[production] |
| 19:20 |
<mutante> |
testreduce1001 - re-enabled puppet, confirmed parsoid-rt service was now stopped properly by puppet while it runs as before on scandium, the previous parsoid-testing host. switching it over is now a Hiera one-liner. (T257906) |
[production] |
| 19:15 |
<twentyafterfour@deploy1001> |
Synchronized php: group1 wikis to 1.36.0-wmf.5 refs T257973 (duration: 01m 04s) |
[production] |
| 19:14 |
<twentyafterfour@deploy1001> |
rebuilt and synchronized wikiversions files: group1 wikis to 1.36.0-wmf.5 refs T257973 |
[production] |
| 19:02 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: 60af096b80a8ef7bc94ec40ce203fd27b0c97f26: Add autopatrolled group at arzwiki (T260761) (duration: 01m 04s) |
[production] |
| 18:52 |
<mutante> |
testreduce1001 - disable puppet; stop parsoid-rt service |
[production] |
| 18:47 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: 924a03bd624d6750a7e776e09713056cc45e5cc5: Add clinton.presidentiallibraries.us to the wgCopyUploadsDomains allowlist of Wikimedia Commons (T259927) (duration: 01m 04s) |
[production] |
| 18:45 |
<urbanecm@deploy1001> |
Synchronized wmf-config/CommonSettings.php: 83b34e1bd1ed804a70f67e089580e082f89e2a0f: ClosedWikiProvider: Use testUserForCreation rather than testForAuthentication (T258695) (duration: 01m 04s) |
[production] |
| 18:39 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: 95d45f6e002df78d4860a711042d77a6b0bdecb9: Dont index Draft (118) and Draft talk (119) on hywiki (T260804) (duration: 01m 04s) |
[production] |
| 18:32 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: 803cb1a0d2c8cc6df8e4e88ab3c4d27eb71d01b3: Update taglines for various projects (T258552) (duration: 01m 04s) |
[production] |
| 18:30 |
<urbanecm@deploy1001> |
Synchronized static/images/mobile/copyright/: 803cb1a0d2c8cc6df8e4e88ab3c4d27eb71d01b3: Update taglines for various projects (T258552) (duration: 01m 06s) |
[production] |
| 18:25 |
<mutante> |
rebooting webperf1002 VM on ganeti level (outside OS) to upgrade rom 8 to 16GB RAM (T260192) |
[production] |
| 18:24 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 18:24 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 18:24 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: bb4aa44b0bd5b2b33d190d3af81e038e5fc55e3f: Configure namespaces on commons to include categories (T198716) (duration: 01m 04s) |
[production] |
| 18:21 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: b9043331c1c1b352256cffd471b9ff128806607c: Update project wordmarks (T254788; sync 2/2) (duration: 01m 04s) |
[production] |
| 18:19 |
<urbanecm@deploy1001> |
Synchronized static/images/mobile/copyright/: b9043331c1c1b352256cffd471b9ff128806607c: Update project wordmarks (T254788; sync 1/2) (duration: 01m 06s) |
[production] |
| 18:15 |
<mutante> |
rebooting webperf2002 VM on ganeti level (outside OS) to upgrade rom 8 to 16GB RAM (T260192) |
[production] |
| 18:15 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: a6f8354e7599a5e92bea060807065f5b42c540e5: Enable $wgMFNoindexPages for all wikis (T255458) (duration: 01m 07s) |
[production] |
| 18:13 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 18:13 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 18:13 |
<ppchelko@deploy1001> |
helmfile [staging] Ran 'sync' command on namespace 'api-gateway' for release 'staging' . |
[production] |
| 17:38 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) |
[production] |
| 17:38 |
<mutante> |
decom'ing releases2001.codfw.wmnet ( |
[production] |
| 17:37 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 16:39 |
<ppchelko@deploy1001> |
helmfile [staging] Ran 'sync' command on namespace 'api-gateway' for release 'staging' . |
[production] |
| 16:37 |
<ppchelko@deploy1001> |
helmfile [staging] Ran 'sync' command on namespace 'api-gateway' for release 'staging' . |
[production] |
| 16:32 |
<andrew@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 16:30 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 15:41 |
<rzl> |
finished exercising the switchdc cookbooks with --live-test for now, all changes reverted including re-enabling puppet on cumin1001 |
[production] |
| 15:38 |
<rzl@cumin1001> |
END (PASS) - Cookbook sre.switchdc.mediawiki.08-start-maintenance (exit_code=0) |
[production] |
| 15:37 |
<rzl@cumin1001> |
START - Cookbook sre.switchdc.mediawiki.08-start-maintenance |
[production] |
| 15:34 |
<rzl@cumin1001> |
END (PASS) - Cookbook sre.switchdc.mediawiki.08-restore-ttl (exit_code=0) |
[production] |
| 15:34 |
<rzl@cumin1001> |
START - Cookbook sre.switchdc.mediawiki.08-restore-ttl |
[production] |
| 15:33 |
<rzl@cumin1001> |
END (FAIL) - Cookbook sre.switchdc.mediawiki.08-restore-ttl (exit_code=99) |
[production] |
| 15:33 |
<rzl@cumin1001> |
START - Cookbook sre.switchdc.mediawiki.08-restore-ttl |
[production] |
| 15:31 |
<jbond42> |
update java.security https://gerrit.wikimedia.org/r/c/operations/puppet/+/593467 |
[production] |
| 15:30 |
<oblivian@cumin1001> |
conftool action : set/ttl=300; selector: dnsdisc=api-rw |
[production] |
| 15:26 |
<rzl@cumin1001> |
END (FAIL) - Cookbook sre.switchdc.mediawiki.00-reduce-ttl (exit_code=99) |
[production] |
| 15:26 |
<rzl@cumin1001> |
START - Cookbook sre.switchdc.mediawiki.00-reduce-ttl |
[production] |
| 15:22 |
<rzl@cumin1001> |
END (FAIL) - Cookbook sre.switchdc.mediawiki.08-restore-ttl (exit_code=99) |
[production] |
| 15:22 |
<rzl@cumin1001> |
START - Cookbook sre.switchdc.mediawiki.08-restore-ttl |
[production] |
| 15:18 |
<godog> |
prometheus codfw lvextend --resizefs --size +80G /dev/mapper/vg--ssd-prometheus--ops |
[production] |
| 15:17 |
<rzl@cumin1001> |
END (PASS) - Cookbook sre.switchdc.mediawiki.00-reduce-ttl (exit_code=0) |
[production] |