|
2018-09-19
§
|
| 07:12 |
<marostegui> |
Disable puppet on databases to test new alerts - T200509 https://phabricator.wikimedia.org/T172489 |
[production] |
| 07:10 |
<jynus> |
restart db1109 for upgrade |
[production] |
| 06:56 |
<jynus@deploy1001> |
Synchronized wmf-config/db-codfw.php: Repool db2041 with low load (duration: 00m 54s) |
[production] |
| 06:00 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Repool db2036 (duration: 00m 58s) |
[production] |
| 05:53 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Depool db2036 (duration: 00m 57s) |
[production] |
| 05:49 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Repool db2050 (duration: 00m 58s) |
[production] |
| 05:41 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Depool db2050 (duration: 00m 57s) |
[production] |
| 05:37 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Repool db2074 (duration: 00m 57s) |
[production] |
| 05:31 |
<marostegui> |
Drop echo tables from s3 db2074 - T153638 |
[production] |
| 05:31 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Depool db2074 (duration: 00m 57s) |
[production] |
| 05:25 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Repool db2057 (duration: 00m 58s) |
[production] |
| 05:19 |
<marostegui> |
Drop echo tables from s7:kowiki - T153638 |
[production] |
| 05:16 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Depool db2057 (duration: 02m 13s) |
[production] |
| 02:34 |
<twentyafterfour@deploy1001> |
Finished scap: SWAT: full sync to update l10n for https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/WikimediaMessages/+/461230/ (duration: 61m 20s) |
[production] |
| 01:33 |
<twentyafterfour@deploy1001> |
Started scap: SWAT: full sync to update l10n for https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/WikimediaMessages/+/461230/ |
[production] |
| 01:28 |
<twentyafterfour@deploy1001> |
Synchronized php-1.32.0-wmf.22/resources/src/mediawiki.util.js: SWAT: sync https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/461227/ (duration: 01m 00s) |
[production] |
|
2018-09-18
§
|
| 23:21 |
<twentyafterfour@deploy1001> |
Synchronized wmf-config/: SWAT: tested on mwdebug1001, now syncing wmf-config settings to the whole cluster (duration: 00m 59s) |
[production] |
| 23:15 |
<twentyafterfour> |
SWAT: deploying 3 patches for ebernhardson: 77deae12f 7009fe473 and 12cc420fc |
[production] |
| 22:10 |
<bblack> |
depooled ulsfo, some uknown router issue |
[production] |
| 21:42 |
<bblack> |
authdns1001 seems stable/fine running gdnsd-2.99.9-beta so far. If issues crop up later, don't hesitate to (a) downgrade back to stretch-backports gdnsd-2.3.0-1~bpo9+1 or (b) call me! |
[production] |
| 21:25 |
<bblack> |
authdns1001: testing gdnsd version update (2.99.9-beta) |
[production] |
| 20:58 |
<catrope@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Set ActorTableSchemaMigrationStage back to OLD on test wikis, mediawikiwiki (T188327, T204669) (duration: 00m 57s) |
[production] |
| 20:57 |
<catrope@deploy1001> |
sync-file aborted: Set ActorTableSchemaMigrationStage back to OLD on test wikis, mediawikiwiki (duration: 00m 10s) |
[production] |
| 20:05 |
<hashar@deploy1001> |
Synchronized php-1.32.0-wmf.22/extensions/OAuth/frontend/specialpages/SpecialMWOAuthListConsumers.php: Fix escapeForHtml method name - T204757 (duration: 00m 58s) |
[production] |
| 19:56 |
<ppchelko@deploy1001> |
Finished deploy [cpjobqueue/deploy@58f9ed3]: Log errors for HTTP error T203929 (duration: 00m 49s) |
[production] |
| 19:55 |
<ppchelko@deploy1001> |
Started deploy [cpjobqueue/deploy@58f9ed3]: Log errors for HTTP error T203929 |
[production] |
| 19:43 |
<bblack> |
uploaded gdnsd-2.99.9-beta1-1+wmf1 to reprepro for stretch-wikimedia |
[production] |
| 19:20 |
<hashar@deploy1001> |
rebuilt and synchronized wikiversions files: group0 to 1.32.0-wmf.22 |
[production] |
| 19:18 |
<ppchelko@deploy1001> |
Finished deploy [restbase/deploy@4c3128f] (dev-cluster): Remove restrictions table T203835, Split 25% PDF traffic to Proton T186748, Metrics endpoints improvements, don't monitor PDF endpoints, take 2, feed timing out (duration: 06m 32s) |
[production] |
| 19:11 |
<ppchelko@deploy1001> |
Started deploy [restbase/deploy@4c3128f] (dev-cluster): Remove restrictions table T203835, Split 25% PDF traffic to Proton T186748, Metrics endpoints improvements, don't monitor PDF endpoints, take 2, feed timing out |
[production] |
| 18:49 |
<ppchelko@deploy1001> |
Finished deploy [restbase/deploy@4c3128f]: Remove restrictions table T203835, Split 25% PDF traffic to Proton T186748, Metrics endpoints improvements, don't monitor PDF endpoints, take 2, feed timing out (duration: 06m 14s) |
[production] |
| 18:43 |
<ppchelko@deploy1001> |
Started deploy [restbase/deploy@4c3128f]: Remove restrictions table T203835, Split 25% PDF traffic to Proton T186748, Metrics endpoints improvements, don't monitor PDF endpoints, take 2, feed timing out |
[production] |
| 18:40 |
<ppchelko@deploy1001> |
Finished deploy [restbase/deploy@4c3128f]: Remove restrictions table T203835, Split 25% PDF traffic to Proton T186748, Metrics endpoints improvements, don't monitor PDF endpoints, take 2, feed timing out (duration: 07m 58s) |
[production] |
| 18:32 |
<ppchelko@deploy1001> |
Started deploy [restbase/deploy@4c3128f]: Remove restrictions table T203835, Split 25% PDF traffic to Proton T186748, Metrics endpoints improvements, don't monitor PDF endpoints, take 2, feed timing out |
[production] |
| 18:31 |
<ppchelko@deploy1001> |
Finished deploy [restbase/deploy@4c3128f]: Remove restrictions table T203835, Split 25% PDF traffic to Proton T186748, Metrics endpoints improvements, don't monitor PDF endpoints (duration: 08m 55s) |
[production] |
| 18:22 |
<ppchelko@deploy1001> |
Started deploy [restbase/deploy@4c3128f]: Remove restrictions table T203835, Split 25% PDF traffic to Proton T186748, Metrics endpoints improvements, don't monitor PDF endpoints |
[production] |
| 18:08 |
<ppchelko@deploy1001> |
Finished deploy [restbase/deploy@55100d4]: Remove restrictions table T203835, Split 25% PDF traffic to Proton T186748, Metrics endpoints improvements (duration: 03m 37s) |
[production] |
| 18:04 |
<ppchelko@deploy1001> |
Started deploy [restbase/deploy@55100d4]: Remove restrictions table T203835, Split 25% PDF traffic to Proton T186748, Metrics endpoints improvements |
[production] |
| 17:29 |
<mutante> |
scandium - move from role(spare) to role(parsoid_testing), making it equal to ruthenium (T201366) |
[production] |
| 16:45 |
<addshore@deploy1001> |
Synchronized php-1.32.0-wmf.22/includes/watcheditem/WatchedItemStore.php: [[gerrit:461155|WatchedItemStore::countVisitingWatchersMultiple() fix]] T204729 (duration: 00m 57s) |
[production] |
| 16:43 |
<addshore@deploy1001> |
Synchronized php-1.32.0-wmf.20/includes/watcheditem/WatchedItemStore.php: [[gerrit:461154|WatchedItemStore::countVisitingWatchersMultiple() fix]] T204729 (duration: 00m 59s) |
[production] |
| 16:33 |
<XioNoX> |
delete `filter common-infrastructure4` on cr1/2-eqiad, unused/obsolete after T198623 |
[production] |
| 16:33 |
<mutante> |
mwmaint2001 - nagios-nrpe-server had status 'failed' and caused all NRPE Icinga checks to fail but recovered after simply starting it again |
[production] |
| 16:32 |
<mutante> |
radon - re-enabled disabled puppet without reason (decom) T202040 |
[production] |
| 16:32 |
<mutante> |
mwmaint2001 - starting nagios-nrpe-server |
[production] |
| 16:22 |
<XioNoX> |
delete `term cumin` from cr1/2-eqiad analytics filter (already permited by established-tcp term) |
[production] |
| 16:02 |
<moritzm> |
installing policykit-1 security updates on jessie |
[production] |
| 15:55 |
<elukey@deploy1001> |
Finished deploy [analytics/refinery@1a6235a]: Fix cron scrips from the NYC offsite (duration: 09m 32s) |
[production] |
| 15:53 |
<XioNoX> |
add cumin2001 to mr* security policies - T204730 |
[production] |
| 15:47 |
<moritzm> |
installing spice security updates |
[production] |