|
2017-03-01
§
|
| 13:16 |
<marostegui> |
run pt-table-checksum on idwiki - T154485 |
[production] |
| 12:43 |
<moritzm> |
installing apache2 security updates on mw1261 |
[production] |
| 12:22 |
<godog> |
upgrade thumbor to 0.1.13 on thumbor100[12] |
[production] |
| 12:21 |
<hashar> |
deployment-prep cleaning out git repos on deployment-tin |
[releng] |
| 11:32 |
<jynus> |
running alter table on db2037 T147747 |
[production] |
| 11:27 |
<moritzm> |
upgrading nginx on meiterium/archiva.wikimedia.org to 1.11.4 (using openssl 1.1) |
[production] |
| 11:02 |
<moritzm> |
uploaded lz4 0.0~r131 for jessie-wikimedia to apt.wikimedia.org (required by HHVM 3.18) |
[production] |
| 10:16 |
<joal> |
Kill and restart webrequest-load-maps coordinator checking for new oozie_loader_memory parameter (starting from 2017-02-28T18:00 - 2g launcher memory) |
[analytics] |
| 10:00 |
<legoktm> |
deployed https://gerrit.wikimedia.org/r/340280 to slaves |
[releng] |
| 09:39 |
<joal> |
Kill and restart webrequest-load-maps coordinator checking for new oozie_loader_memory parameter (starting from 2017-02-28T18:00) |
[analytics] |
| 09:33 |
<jynus> |
running alter table on db1037 T147747 |
[production] |
| 09:20 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1056 after maintenance (duration: 00m 41s) |
[production] |
| 09:14 |
<marostegui> |
Deploy alter table s3 (all wikis) user_groups table - T155605 |
[production] |
| 08:49 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1026 after maintenance (duration: 00m 40s) |
[production] |
| 08:18 |
<moritzm> |
installing libgd2 security updates on trusty (jessie already fixed) |
[production] |
| 07:17 |
<elukey> |
restarted manually the browser-general-coord failed jobs |
[analytics] |
| 07:13 |
<elukey> |
restarted manually the pageview-hourly-coord failed jobs |
[analytics] |
| 07:09 |
<elukey> |
restarted manually the pageview-druid-monthly-coord (february job failed) |
[analytics] |
| 07:06 |
<elukey> |
restarted manually via Hue UI the webrequest-load-coord-misc failed jobs |
[analytics] |
| 07:05 |
<marostegui> |
Deploy alter table enwiki.revision - dbstore2002 - T132416 |
[production] |
| 06:59 |
<elukey> |
restarted manually via Hue UI the webrequest-load-coord-maps failed jobs |
[analytics] |
| 04:28 |
<Krinkle> |
Reloading Zuul to deploy https://gerrit.wikimedia.org/r/340465 |
[releng] |
| 03:06 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Wed Mar 1 03:06:24 UTC 2017 (duration 5m 46s) |
[production] |
| 03:00 |
<l10nupdate@tin> |
scap sync-l10n completed (1.29.0-wmf.14) (duration: 13m 51s) |
[production] |
| 02:29 |
<l10nupdate@tin> |
scap sync-l10n completed (1.29.0-wmf.13) (duration: 08m 03s) |
[production] |
| 01:03 |
<Reedy> |
beta-scap-eqiad giving Host key verification failed |
[releng] |
| 01:00 |
<reedy@tin> |
Synchronized wmf-config/InitialiseSettings.php: Remove DonationInterface loading as gone from master (primarily to unbreak beta) (duration: 00m 42s) |
[production] |
| 00:59 |
<eileen1> |
Update CiviCRM from 04b49b0393295610c530dc29cb0218e6755ef24f to b3f6eef513cf2ea2c48379f9e0bcd4b02b18e379 |
[production] |
| 00:59 |
<reedy@tin> |
Synchronized wmf-config/CommonSettings.php: Remove DonationInterface loading as gone from master (primarily to unbreak beta) (duration: 00m 40s) |
[production] |
|
2017-02-28
§
|
| 22:29 |
<mutante> |
(T157675) - delete salt keys - [neodymium:~] $ for mcnode in $(seq 2001 2016); do sudo salt-key -d mc${mcnode}.codfw.wmnet; done |
[production] |
| 22:26 |
<mutante> |
(T157675) - revoke puppet certs, deactivate nodes, rm from icinga. [puppetmaster1001:~] $ for mcnode in $(seq 2001 2016); do puppet node clean mc${mcnode}.codfw.wmnet && puppet node deactivate mc${mcnode}.codfw.wmnet ; done |
[production] |
| 21:58 |
<awight> |
update payments from 2a0c3b2148805fe258f124a700e7f3c34f306783 to 66d812517acc47a8a7e07b5fe269950b21c86be3 |
[production] |
| 21:51 |
<eileen1> |
update CiviCRM from a2875c55a5760c375b133c71a7ec22846b4d68c3 to 04b49b0393295610c530dc29cb0218e6755ef24f |
[production] |
| 21:44 |
<urandom> |
Updating RESTBase mobileapps tables (all remaining) to use time-windowed compaction |
[production] |
| 21:40 |
<maxsem@tin> |
Finished deploy [kartotherian/deploy@81db48c]: Second attempt at 81db48c6db1fdfea81a01796ccbb7cfeccc43052 (duration: 06m 39s) |
[production] |
| 21:34 |
<maxsem@tin> |
Started deploy [kartotherian/deploy@81db48c]: Second attempt at 81db48c6db1fdfea81a01796ccbb7cfeccc43052 |
[production] |
| 21:23 |
<MaxSem> |
Completely disabled kartotherian on maps-test2004, it just logs errors |
[production] |
| 21:05 |
<_joe_> |
manually installing nodejs on wasat T156922 |
[production] |
| 20:50 |
<maxsem@tin> |
Synchronized wmf-config/wikitech.php: https://gerrit.wikimedia.org/r/#/c/340357/2 (duration: 00m 40s) |
[production] |
| 20:33 |
<urandom> |
Updating RESTBase mobileapps tables (phase0) to use time-windowed compaction |
[production] |
| 20:30 |
<demon@tin> |
Synchronized wmf-config/wikitech.php: no moar forms on wikitech (duration: 00m 39s) |
[production] |
| 20:03 |
<demon@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: group0 to wmf.14 |
[production] |
| 19:43 |
<thcipriani> |
deployment-puppetmaster02 puppetmaster running again, apache2 was refusing to start with: Invalid command 'SSLOpenSSLConfCmd' -- installed apache from wmf repo instead of debian fixed it |
[releng] |
| 19:34 |
<demon@tin> |
Synchronized php-1.29.0-wmf.14/extensions/WikimediaEvents/modules/ext.wikimediaEvents.geoFeatures.js: Roan made me do it (duration: 00m 39s) |
[production] |
| 19:26 |
<demon@tin> |
Finished scap: testwiki to wmf.14 + l10n bootstrap (duration: 55m 14s) |
[production] |
| 19:04 |
<urandom> |
Updating RESTBase mobileapps tables (wikimedia) to uses time-windowed compaction |
[production] |
| 18:31 |
<demon@tin> |
Started scap: testwiki to wmf.14 + l10n bootstrap |
[production] |
| 18:03 |
<joal> |
restart pageview oozie job for 2017-02-28T12:00 |
[analytics] |
| 17:53 |
<elukey> |
restarted via Hue Feb 2017 14:00:00 webrequest-load-coord-misc/maps |
[analytics] |
| 17:43 |
<urandom> |
Updating RESTBase mobileapps tables (wikipedia) to uses time-windowed compaction |
[production] |