|
2018-09-20
§
|
| 07:25 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool pc1004 (duration: 00m 57s) |
[production] |
| 07:15 |
<marostegui> |
Stop MySQL on pc1004 for kernel upgrade |
[production] |
| 07:14 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool pc1004 (duration: 00m 57s) |
[production] |
| 06:50 |
<eileen> |
process-control config revision is 3069f85578 |
[production] |
| 06:42 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Repool db2070 (duration: 00m 58s) |
[production] |
| 06:25 |
<mobrovac@deploy1001> |
Started restart [cpjobqueue/deploy@58f9ed3]: Reset the Kafka connections |
[production] |
| 06:20 |
<mobrovac@deploy1001> |
Synchronized rpc/RunSingleJob.php: Have RunSingleJob send the X-Readonly header - T204154 (duration: 00m 58s) |
[production] |
| 06:09 |
<marostegui> |
Deploy schema change on db2070 - T203709 |
[production] |
| 06:07 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Depool db2070 (duration: 00m 58s) |
[production] |
| 05:54 |
<marostegui> |
Deploy schema change on s3:mediawikiwiki for echo tables codfw - T51593 |
[production] |
| 05:49 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Resume reads on db2034 (duration: 00m 57s) |
[production] |
| 05:34 |
<marostegui> |
Deploy schema change on db2034 (x1 master) |
[production] |
| 05:33 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Stop reads on db2034 (duration: 00m 57s) |
[production] |
| 05:28 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Repool db2069 after alter table (duration: 00m 57s) |
[production] |
| 05:12 |
<marostegui> |
Deploy schema change on db2069 |
[production] |
| 05:11 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Depool db2069 for alter table (duration: 01m 00s) |
[production] |
| 03:06 |
<l10nupdate@deploy1001> |
ResourceLoader cache refresh completed at Thu Sep 20 03:06:52 UTC 2018 (duration 10m 47s) |
[production] |
| 02:56 |
<l10nupdate@deploy1001> |
scap sync-l10n completed (1.32.0-wmf.22) (duration: 13m 54s) |
[production] |
| 02:23 |
<l10nupdate@deploy1001> |
scap sync-l10n completed (1.32.0-wmf.20) (duration: 09m 26s) |
[production] |
| 01:27 |
<krinkle@deploy1001> |
Synchronized php-1.32.0-wmf.22/resources/src/startup/: I6c77b25856 (duration: 00m 55s) |
[production] |
| 01:14 |
<krinkle@deploy1001> |
Synchronized php-1.32.0-wmf.22/extensions/GlobalPreferences/includes/GlobalPreferencesFactory.php: I52434a523f60e, T204864 (duration: 00m 58s) |
[production] |
| 00:52 |
<krinkle@deploy1001> |
Synchronized php-1.32.0-wmf.22/extensions/3D/modules/: I9e718957a497, T204621 (duration: 00m 58s) |
[production] |
| 00:49 |
<mutante> |
temp disabled puppet on mw2*, deployed gerrit 461393, confirmed noop, re-enabled puppet on mw2* |
[production] |
|
2018-09-19
§
|
| 23:21 |
<catrope@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Remove obsolete $wgPopupsBetaFeature, part 2 (T203589) (duration: 00m 56s) |
[production] |
| 23:21 |
<catrope@deploy1001> |
Synchronized wmf-config/CommonSettings.php: Remove obsolete $wgPopupsBetaFeature, part 1 (T203589) (duration: 00m 56s) |
[production] |
| 23:17 |
<catrope@deploy1001> |
Synchronized php-1.32.0-wmf.22/extensions/PageTriage/: PageTriage fixes for T203184 (duration: 00m 58s) |
[production] |
| 23:14 |
<catrope@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Enable Page issues A/B test on lvwiki (T204609) (duration: 00m 58s) |
[production] |
| 22:00 |
<XioNoX> |
merging icinga check_bfd |
[production] |
| 21:24 |
<jforrester@deploy1001> |
Synchronized php-1.32.0-wmf.22/extensions/VisualEditor/includes/SpecialCollabPad.php: T204873 Fix special page override hot deploy (duration: 00m 57s) |
[production] |
| 20:26 |
<ladsgroup@deploy1001> |
Finished deploy [ores/deploy@76fe25a]: ORES doesn't block hammering IPs (T204862) (duration: 21m 13s) |
[production] |
| 20:24 |
<XioNoX> |
repool ulsfo |
[production] |
| 20:21 |
<ejegg> |
updated fundraising CiviCRM from 2d9360cfac to 7f58670e88 |
[production] |
| 20:11 |
<bsitzmann@deploy1001> |
Finished deploy [mobileapps/deploy@55d0b34]: Update mobileapps to a224e99 Ia80abe02490 (duration: 03m 29s) |
[production] |
| 20:07 |
<bsitzmann@deploy1001> |
Started deploy [mobileapps/deploy@55d0b34]: Update mobileapps to a224e99 Ia80abe02490 |
[production] |
| 20:05 |
<ladsgroup@deploy1001> |
Started deploy [ores/deploy@76fe25a]: ORES doesn't block hammering IPs (T204862) |
[production] |
| 19:40 |
<hashar> |
web request 60 second timeout when deploying is filled as https://phabricator.wikimedia.org/T204871 |
[production] |
| 19:30 |
<hashar> |
while promoting group1 to 1.32.0-wmf.22 lot of web requests timed out at 60 seconds. Roughly from 19:24 to 19:28. But that is no more occurring | T191068 |
[production] |
| 19:25 |
<hashar@deploy1001> |
Synchronized php: group1 wikis to 1.32.0-wmf.22 (duration: 00m 55s) |
[production] |
| 19:24 |
<hashar@deploy1001> |
rebuilt and synchronized wikiversions files: group1 wikis to 1.32.0-wmf.22 |
[production] |
| 19:16 |
<hashar@deploy1001> |
Synchronized php-1.32.0-wmf.22/extensions/Translate/tag/PageTranslationHooks.php: Avoid warnings and errors caused by x-pagetranslation-tag - T204797 (duration: 00m 58s) |
[production] |
| 18:43 |
<gehel> |
starting elasticsearch eqiad cluster restart for new systemd unit |
[production] |
| 18:33 |
<chasemp> |
add security_team_bot to acl*security_team in phab |
[production] |
| 17:52 |
<robh> |
no errors in notebook1003 SEL |
[production] |
| 17:51 |
<robh> |
notebook1003 unresponsive to icinga checks and serial console, rebooting |
[production] |
| 17:31 |
<mutante> |
notebook1003 - starting failed nagios-nrpe-server |
[production] |
| 17:20 |
<cmjohnson1> |
powering off mw1254 to reseat DIMM T204491 |
[production] |
| 17:03 |
<mobrovac@deploy1001> |
Finished deploy [restbase/deploy@b35727e]: Bug fix: return the stored headers in the key rev value bucket, take #4 (duration: 09m 03s) |
[production] |
| 16:54 |
<mobrovac@deploy1001> |
Started deploy [restbase/deploy@b35727e]: Bug fix: return the stored headers in the key rev value bucket, take #4 |
[production] |
| 16:54 |
<mobrovac@deploy1001> |
Finished deploy [restbase/deploy@b35727e]: Bug fix: return the stored headers in the key rev value bucket, take #3 (duration: 05m 28s) |
[production] |
| 16:52 |
<gehel> |
rolling restart of elasticsearch / logstash for new systemd unit completed - new systemd unit is elasticsearch_5@production-logstash-eqiad |
[production] |