|
2017-10-24
§
|
| 10:50 |
<dcausse> |
cirrus/elasticsearch: reindexing of 167 small wikis done (T177871) |
[production] |
| 09:22 |
<marostegui> |
Stop s1 on db2092 to copy its data to db2088 - T178359 |
[production] |
| 08:35 |
<hashar> |
beta: cherry pick https://gerrit.wikimedia.org/r/#/c/386077/4 "hieradata for varnish caches" - T178841 |
[releng] |
| 08:26 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Restore db1082 original weight - T178460 (duration: 00m 45s) |
[production] |
| 07:48 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Increase db1082 weight - T178460 (duration: 00m 45s) |
[production] |
| 07:46 |
<marostegui> |
Stop replication in sync on db1103 and db2018 to fix data drifts - T164488 |
[production] |
| 07:40 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1078 - T164488 (duration: 00m 46s) |
[production] |
| 07:28 |
<marostegui> |
Stop replication in sync on db1078 and db1103 to fix data drifts - https://phabricator.wikimedia.org/T164488 |
[production] |
| 06:22 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1078 - T164488 (duration: 00m 45s) |
[production] |
| 06:16 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1082 with low weight - T178460 (duration: 00m 47s) |
[production] |
| 02:32 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Tue Oct 24 02:32:25 UTC 2017 (duration 6m 33s) |
[production] |
| 02:25 |
<l10nupdate@tin> |
scap sync-l10n completed (1.31.0-wmf.4) (duration: 08m 13s) |
[production] |
|
2017-10-23
§
|
| 23:09 |
<demon@tin> |
Synchronized wmf-config/throttle.php: Working Class Movement Library (Salford) throttle rule (duration: 00m 46s) |
[production] |
| 22:59 |
<bblack> |
cp4024: varnish-backend-restart for lag |
[production] |
| 21:40 |
<XioNoX> |
increasing MTU on Zayo transit interfaces |
[production] |
| 20:33 |
<arlolra> |
Updated Parsoid to 1cefef12 (T178217) |
[production] |
| 20:29 |
<Krinkle> |
Puppet still failing, now with: "Error 400 on SERVER: Could not find data item cache::fe_transient_gb in any Hiera data file and no default supplied at /etc/puppet/modules/profile/manifests/cache/text.pp:12 on node deployment-cache-text04.deployment-prep.eqiad.wmflabs" |
[releng] |
| 20:29 |
<Krinkle> |
Previous edit failed. Horizon saved the field as blank. Presumably because the class is unknown in the current version of puppet manifests it has. Strange that it normalises in this way. |
[releng] |
| 20:28 |
<Krinkle> |
Edit horizon "Other classes" config for deployment-prep/deployment-cache-text04. Rename role::prometheus::varnish_exporter to profile::prometheus::varnish_exporter |
[releng] |
| 20:25 |
<arlolra@tin> |
Finished deploy [parsoid/deploy@0759bd1]: Updating Parsoid to 1cefef12 (duration: 11m 14s) |
[production] |
| 20:14 |
<arlolra@tin> |
Started deploy [parsoid/deploy@0759bd1]: Updating Parsoid to 1cefef12 |
[production] |
| 20:13 |
<Krinkle> |
Puppet run still failing on Beta cluster varnish: "Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Could not find class role::prometheus::varnish_exporter" |
[releng] |
| 19:37 |
<bsitzmann@tin> |
Finished deploy [mobileapps/deploy@a654ce1]: Update mobileapps to 3628105 (T178828) (duration: 06m 59s) |
[production] |
| 19:30 |
<bsitzmann@tin> |
Started deploy [mobileapps/deploy@a654ce1]: Update mobileapps to 3628105 (T178828) |
[production] |
| 19:00 |
<joal@tin> |
Finished deploy [analytics/aqs/deploy@ad22e0c]: Upgrade AQS node modules to restbase-latests - Redeploy after config fix (duration: 02m 34s) |
[production] |
| 18:58 |
<joal@tin> |
Started deploy [analytics/aqs/deploy@ad22e0c]: Upgrade AQS node modules to restbase-latests - Redeploy after config fix |
[production] |
| 18:58 |
<joal@tin> |
Finished deploy [analytics/aqs/deploy@ad22e0c]: Upgrade AQS node modules to restbase-latests - Redeploy after config fix (duration: 05m 07s) |
[production] |
| 18:52 |
<joal@tin> |
Started deploy [analytics/aqs/deploy@ad22e0c]: Upgrade AQS node modules to restbase-latests - Redeploy after config fix |
[production] |
| 18:52 |
<joal@tin> |
Finished deploy [analytics/aqs/deploy@ad22e0c]: Upgrade AQS node modules to restbase-latests (duration: 321m 51s) |
[production] |
| 18:50 |
<joal> |
Deploying AQS after fix |
[analytics] |
| 17:56 |
<gehel@tin> |
Finished deploy [wdqs/wdqs@b84592f]: WDQS GUI update (duration: 01m 58s) |
[production] |
| 17:54 |
<gehel@tin> |
Started deploy [wdqs/wdqs@b84592f]: WDQS GUI update |
[production] |
| 17:48 |
<XenoRyet> |
updated smashpig from d056a131ccbbd4b505da4f99e04f5337fdd9c32e to 5262d532e30aef76a72fd9b8ff248e9a9570f8f7 |
[production] |
| 17:32 |
<demon@tin> |
Synchronized scap/plugins/prep.py: Fixes and improvements of various sorts (duration: 00m 45s) |
[production] |
| 17:26 |
<gehel@tin> |
Finished deploy [wdqs/wdqs@4626acb]: deploying latest WDQS version, blazegraph + GUI updates (duration: 02m 07s) |
[production] |
| 17:24 |
<gehel@tin> |
Started deploy [wdqs/wdqs@4626acb]: deploying latest WDQS version, blazegraph + GUI updates |
[production] |
| 16:53 |
<demon@tin> |
Pruned MediaWiki: 1.30.0-wmf.19 (duration: 03m 09s) |
[production] |
| 15:59 |
<elukey> |
forced BBU learn cycle on db1046 - T166141 |
[production] |
| 14:59 |
<herron> |
wiped resolver cache of records puppet.cofdw.wmnet and puppet.ulsfo.wmnet (T177254) |
[production] |
| 14:57 |
<bd808> |
Started webservice |
[tools.wsexport] |
| 14:49 |
<chasemp> |
wall message and scheduled reboot in 5m for bastion-03 |
[tools] |
| 14:46 |
<herron> |
depooling codfw puppetmaster (via dns) |
[production] |
| 14:31 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1078 - T164488 (duration: 00m 46s) |
[production] |
| 14:05 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Repool db2037 - T178359 (duration: 00m 46s) |
[production] |
| 13:48 |
<marostegui> |
Stop mysql and poweroff db1082 for HW maintenance - T178460 |
[production] |
| 13:30 |
<joal> |
deploy AQS from tin |
[analytics] |
| 13:30 |
<joal@tin> |
Started deploy [analytics/aqs/deploy@ad22e0c]: Upgrade AQS node modules to restbase-latests |
[production] |
| 13:19 |
<hashar@tin> |
Synchronized wmf-config/InitialiseSettings.php: Add aleph500.biblacad.ro to $wgCopyUploadsDomains - T178753 (duration: 00m 47s) |
[production] |
| 13:19 |
<gehel> |
rolling restart of wdqs for GC tuning - T175919 |
[production] |
| 13:18 |
<hashar@tin> |
Synchronized wmf-config/InitialiseSettings.php: Remove hooserv.net from wgCopyUploadsDomains (duration: 00m 47s) |
[production] |