|
2017-10-23
ยง
|
| 23:09 |
<demon@tin> |
Synchronized wmf-config/throttle.php: Working Class Movement Library (Salford) throttle rule (duration: 00m 46s) |
[production] |
| 22:59 |
<bblack> |
cp4024: varnish-backend-restart for lag |
[production] |
| 21:40 |
<XioNoX> |
increasing MTU on Zayo transit interfaces |
[production] |
| 20:33 |
<arlolra> |
Updated Parsoid to 1cefef12 (T178217) |
[production] |
| 20:29 |
<Krinkle> |
Puppet still failing, now with: "Error 400 on SERVER: Could not find data item cache::fe_transient_gb in any Hiera data file and no default supplied at /etc/puppet/modules/profile/manifests/cache/text.pp:12 on node deployment-cache-text04.deployment-prep.eqiad.wmflabs" |
[releng] |
| 20:29 |
<Krinkle> |
Previous edit failed. Horizon saved the field as blank. Presumably because the class is unknown in the current version of puppet manifests it has. Strange that it normalises in this way. |
[releng] |
| 20:28 |
<Krinkle> |
Edit horizon "Other classes" config for deployment-prep/deployment-cache-text04. Rename role::prometheus::varnish_exporter to profile::prometheus::varnish_exporter |
[releng] |
| 20:25 |
<arlolra@tin> |
Finished deploy [parsoid/deploy@0759bd1]: Updating Parsoid to 1cefef12 (duration: 11m 14s) |
[production] |
| 20:14 |
<arlolra@tin> |
Started deploy [parsoid/deploy@0759bd1]: Updating Parsoid to 1cefef12 |
[production] |
| 20:13 |
<Krinkle> |
Puppet run still failing on Beta cluster varnish: "Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Could not find class role::prometheus::varnish_exporter" |
[releng] |
| 19:37 |
<bsitzmann@tin> |
Finished deploy [mobileapps/deploy@a654ce1]: Update mobileapps to 3628105 (T178828) (duration: 06m 59s) |
[production] |
| 19:30 |
<bsitzmann@tin> |
Started deploy [mobileapps/deploy@a654ce1]: Update mobileapps to 3628105 (T178828) |
[production] |
| 19:00 |
<joal@tin> |
Finished deploy [analytics/aqs/deploy@ad22e0c]: Upgrade AQS node modules to restbase-latests - Redeploy after config fix (duration: 02m 34s) |
[production] |
| 18:58 |
<joal@tin> |
Started deploy [analytics/aqs/deploy@ad22e0c]: Upgrade AQS node modules to restbase-latests - Redeploy after config fix |
[production] |
| 18:58 |
<joal@tin> |
Finished deploy [analytics/aqs/deploy@ad22e0c]: Upgrade AQS node modules to restbase-latests - Redeploy after config fix (duration: 05m 07s) |
[production] |
| 18:52 |
<joal@tin> |
Started deploy [analytics/aqs/deploy@ad22e0c]: Upgrade AQS node modules to restbase-latests - Redeploy after config fix |
[production] |
| 18:52 |
<joal@tin> |
Finished deploy [analytics/aqs/deploy@ad22e0c]: Upgrade AQS node modules to restbase-latests (duration: 321m 51s) |
[production] |
| 18:50 |
<joal> |
Deploying AQS after fix |
[analytics] |
| 17:56 |
<gehel@tin> |
Finished deploy [wdqs/wdqs@b84592f]: WDQS GUI update (duration: 01m 58s) |
[production] |
| 17:54 |
<gehel@tin> |
Started deploy [wdqs/wdqs@b84592f]: WDQS GUI update |
[production] |
| 17:48 |
<XenoRyet> |
updated smashpig from d056a131ccbbd4b505da4f99e04f5337fdd9c32e to 5262d532e30aef76a72fd9b8ff248e9a9570f8f7 |
[production] |
| 17:32 |
<demon@tin> |
Synchronized scap/plugins/prep.py: Fixes and improvements of various sorts (duration: 00m 45s) |
[production] |
| 17:26 |
<gehel@tin> |
Finished deploy [wdqs/wdqs@4626acb]: deploying latest WDQS version, blazegraph + GUI updates (duration: 02m 07s) |
[production] |
| 17:24 |
<gehel@tin> |
Started deploy [wdqs/wdqs@4626acb]: deploying latest WDQS version, blazegraph + GUI updates |
[production] |
| 16:53 |
<demon@tin> |
Pruned MediaWiki: 1.30.0-wmf.19 (duration: 03m 09s) |
[production] |
| 15:59 |
<elukey> |
forced BBU learn cycle on db1046 - T166141 |
[production] |
| 14:59 |
<herron> |
wiped resolver cache of records puppet.cofdw.wmnet and puppet.ulsfo.wmnet (T177254) |
[production] |
| 14:57 |
<bd808> |
Started webservice |
[tools.wsexport] |
| 14:49 |
<chasemp> |
wall message and scheduled reboot in 5m for bastion-03 |
[tools] |
| 14:46 |
<herron> |
depooling codfw puppetmaster (via dns) |
[production] |
| 14:31 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1078 - T164488 (duration: 00m 46s) |
[production] |
| 14:05 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Repool db2037 - T178359 (duration: 00m 46s) |
[production] |
| 13:48 |
<marostegui> |
Stop mysql and poweroff db1082 for HW maintenance - T178460 |
[production] |
| 13:30 |
<joal> |
deploy AQS from tin |
[analytics] |
| 13:30 |
<joal@tin> |
Started deploy [analytics/aqs/deploy@ad22e0c]: Upgrade AQS node modules to restbase-latests |
[production] |
| 13:19 |
<hashar@tin> |
Synchronized wmf-config/InitialiseSettings.php: Add aleph500.biblacad.ro to $wgCopyUploadsDomains - T178753 (duration: 00m 47s) |
[production] |
| 13:19 |
<gehel> |
rolling restart of wdqs for GC tuning - T175919 |
[production] |
| 13:18 |
<hashar@tin> |
Synchronized wmf-config/InitialiseSettings.php: Remove hooserv.net from wgCopyUploadsDomains (duration: 00m 47s) |
[production] |
| 13:08 |
<zeljkof> |
EU SWAT finished, no deploys |
[production] |
| 13:00 |
<bblack@neodymium> |
conftool action : set/pooled=no; selector: dc=ulsfo,cluster=cache_text,name=cp4018.ulsfo.wmnet |
[production] |
| 12:59 |
<bblack@neodymium> |
conftool action : set/pooled=yes; selector: dc=ulsfo,cluster=cache_text,name=cp4032.ulsfo.wmnet |
[production] |
| 12:53 |
<bblack@neodymium> |
conftool action : set/pooled=no; selector: dc=ulsfo,cluster=cache_text,name=cp4017.ulsfo.wmnet |
[production] |
| 12:47 |
<bblack@neodymium> |
conftool action : set/pooled=yes; selector: dc=ulsfo,cluster=cache_text,name=cp4031.ulsfo.wmnet |
[production] |
| 12:46 |
<moritzm> |
upgrading hhvm-wikdiff2 on remaining job runners in codfw |
[production] |
| 12:39 |
<moritzm> |
upgrading hhvm-wikdiff2 on remaining video scalers in codfw |
[production] |
| 12:32 |
<bblack@neodymium> |
conftool action : set/pooled=no; selector: dc=ulsfo,cluster=cache_text,name=cp4010.ulsfo.wmnet |
[production] |
| 12:29 |
<hoo@tin> |
Synchronized wmf-config/InitialiseSettings.php: Remove hooserv.net from wgCopyUploadsDomains (duration: 00m 46s) |
[production] |
| 12:26 |
<moritzm> |
upgrading hhvm-wikdiff2 on mw2153-mw2162 (job runners) |
[production] |
| 12:21 |
<moritzm> |
upgrading hhvm-wikdiff2 on mw2254-mw2258 (app servers) |
[production] |
| 12:20 |
<bblack@neodymium> |
conftool action : set/pooled=yes; selector: dc=ulsfo,cluster=cache_text,name=cp4030.ulsfo.wmnet |
[production] |