1201-1250 of 10000 results (45ms)
2020-02-10 §
09:11 <legoktm> building php74-compile image https://gerrit.wikimedia.org/r/571247 [releng]
09:02 <legoktm> reloading zuul for https://gerrit.wikimedia.org/r/571242 [releng]
08:53 <vgutierrez> pooling cp4029 with buster - T242093 [production]
08:44 <marostegui@cumin1001> dbctl commit (dc=all): 'Increase weight from 1 to 5 for db1107 - T242702', diff saved to https://phabricator.wikimedia.org/P10364 and previous config saved to /var/cache/conftool/dbconfig/20200210-084446-marostegui.json [production]
08:44 <legoktm> reloading zuul for https://gerrit.wikimedia.org/r/539724 [releng]
08:43 <vgutierrez> pooling ncredir2002 with buster - T243391 [production]
08:34 <effie> rolling restart php-fpm on labweb[1001-1002].wikimedia.org,mw*.eqiad.wmnet,scandium.eqiad.wmnet, wtp[1025-1048].eqiad.wmnet [production]
08:32 <effie> update php-apcu on eqiad - T236800 [production]
08:30 <legoktm> reloading zuul for https://gerrit.wikimedia.org/r/571236 [releng]
08:29 <effie> rolling restart php-fpm on cloudweb2001-dev.wikimedia.org,mw[2135-2147,2150-2212,2214-2290].codfw.wmnet,wtp[2001-2020].codfw.wmnet [production]
08:23 <effie> update php-apcu on codfw - T236800 [production]
08:17 <legoktm> updating tox-mysqld docker-pkg image https://gerrit.wikimedia.org/r/571234 [releng]
07:58 <vgutierrez@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
07:56 <vgutierrez@cumin1001> START - Cookbook sre.hosts.downtime [production]
07:54 <moritzm> updating d-i netinst image for Stretch 9.12 point release (which bumped the kernel ABI) [production]
07:43 <legoktm> updating tox docker-pkg image for gettext https://gerrit.wikimedia.org/r/571019 [releng]
07:29 <moritzm> updating d-i netinst image for Buster 10.3 point release (which bumped the kernel ABI) [production]
07:20 <legoktm> updating tox docker-pkg image (try #2) https://gerrit.wikimedia.org/r/571127 [releng]
07:09 <elukey> restore mw1347's mcrouter settings to its default (proxy threads 10 -> 5) [production]
07:01 <marostegui@cumin1001> dbctl commit (dc=all): 'Place db1107 - MariaDB 10.4 on s1 with minimal weight - T242702', diff saved to https://phabricator.wikimedia.org/P10363 and previous config saved to /var/cache/conftool/dbconfig/20200210-070140-marostegui.json [production]
07:00 <legoktm> updating docker-pkg for https://gerrit.wikimedia.org/r/571017 [releng]
06:55 <vgutierrez> depool ncredir2002 and reimage as buster - T243391 [production]
06:53 <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool es1019', diff saved to https://phabricator.wikimedia.org/P10362 and previous config saved to /var/cache/conftool/dbconfig/20200210-065326-marostegui.json [production]
06:51 <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool db1091 T232446', diff saved to https://phabricator.wikimedia.org/P10361 and previous config saved to /var/cache/conftool/dbconfig/20200210-065135-marostegui.json [production]
06:47 <vgutierrez> depool cp4029 & reimage as buster - T242093 [production]
06:45 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool es1019', diff saved to https://phabricator.wikimedia.org/P10360 and previous config saved to /var/cache/conftool/dbconfig/20200210-064553-marostegui.json [production]
06:44 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1091 T232446', diff saved to https://phabricator.wikimedia.org/P10359 and previous config saved to /var/cache/conftool/dbconfig/20200210-064458-marostegui.json [production]
06:39 <marostegui> Compress db1124:3318 - this will generate lag on s8 wiki replicas - T232446 [production]
06:37 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1091 T232446', diff saved to https://phabricator.wikimedia.org/P10358 and previous config saved to /var/cache/conftool/dbconfig/20200210-063716-marostegui.json [production]
06:23 <marostegui> Remove partitions from db1099:3311, db1099:3318 T239453 [production]
06:21 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1099:3318 T239453', diff saved to https://phabricator.wikimedia.org/P10357 and previous config saved to /var/cache/conftool/dbconfig/20200210-062112-marostegui.json [production]
06:18 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool repool db1099:3311 T239453', diff saved to https://phabricator.wikimedia.org/P10356 and previous config saved to /var/cache/conftool/dbconfig/20200210-061822-marostegui.json [production]
06:16 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1091 T232446', diff saved to https://phabricator.wikimedia.org/P10355 and previous config saved to /var/cache/conftool/dbconfig/20200210-061656-marostegui.json [production]
2020-02-09 §
11:52 <wm-bot> <lucaswerkmeister> deployed 97900da63e (another language code improvement) [tools.wd-image-positions]
06:29 <wm-bot> <lucaswerkmeister> deployed 5aa0cf4ccb (better language code handling) [tools.wd-image-positions]
05:11 <cdanis> T238305 hardreset cp3051 [production]
2020-02-08 §
22:53 <Framawiki> qdeled job 2929463 maj_articles_recents stuck for one month [tools.totoazero]
19:12 <_joe_> set cpufreq governor to performance on mw1328 [production]
17:17 <wm-bot> <maurelio> Webservice updated to kubernetes backend running on php7.2 now [tools.stewardbots]
17:14 <wm-bot> <maurelio> Shutting down webservice to upgrade it [tools.stewardbots]
17:04 <_joe_> restarted php7.2-fpm on mw1332 [production]
16:53 <Urbanecm> mwscript resetAuthenticationThrottle.php --wiki=enwiki --signup --ip 12.24.27.50 [production]
16:47 <gjg@deploy1001> Synchronized wmf-config/throttle.php: SWAT: Editathon in Charolette (duration: 00m 58s) [production]
09:35 <elukey> created /wmf/data/raw/wikidata/dumps/all_ttl on hdfs [analytics]
09:35 <elukey> created /wmf/data/raw/wikidata/dumps/all_json on hdfs [analytics]
00:05 <Jeff_Green> switched payments.wikimedia.org to codfw datacenter due to T244610 [production]
2020-02-07 §
23:20 <brennen> Updating dev-images docker-pkg files on contint1001 for T244382 [releng]
23:20 <dpifke> puppetdb on deployment-puppetdb03 was killed by kernel OOM at Feb 7 09:50:29, per syslog. I just ran `systemctl start puppetdb` on that host, to fix puppet issues in beta. [releng]
23:07 <bstorm_> upgraded toollabs-webservice for stetch toolsbeta to 0.60 T244611 [toolsbeta]
22:20 <jeh> ceph: round 2 OSD failover and recovery testing on cloudcephosd1003.wikimedia.org T240718 [production]