1401-1450 of 10000 results (38ms)
2018-03-19 §
09:37 <elukey> restart hadoop daemons on analytics1070 for openjdk upgrades (canary) [production]
09:27 <godog> reimage puppetmaster2001 with stretch - T184562 [production]
09:22 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Slowly repool es1016 after kernel, mariadb and socket location upgrade (duration: 00m 58s) [production]
09:10 <godog> depool codfw puppetmaster - T184562 [production]
09:08 <marostegui> Stop MySQL on es1016 for kernel, mariadb and socket location upgrade [production]
09:07 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool es1016 for kernel, mariadb and socket location upgrade (duration: 00m 58s) [production]
08:57 <moritzm> installing openjdk-8 security updates [production]
08:41 <elukey> reboot thorium for kernel security upgrades (hosts all analytics websites, they will go down temporary) [production]
08:26 <moritzm> installing libvorbis security updates [production]
08:22 <elukey> revert previous state on aqs1004, the new pkg might need some more work - T189529 [production]
08:19 <marostegui> Reset slave on db1106 to get it ready for s1 - https://phabricator.wikimedia.org/T183469 [production]
08:11 <marostegui> Reboot db1106 for kernel upgrade [production]
07:58 <elukey> manually installed cassandra-2.2.6-wmf3 on aqs1004 - T189529 [production]
07:55 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1065 - T183469 (duration: 00m 57s) [production]
07:47 <elukey> drain cassandra instances and reboot aqs1004 for kernel upgrades [production]
07:44 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Move db1106 from s5 to s1 - T183469 (duration: 01m 00s) [production]
07:43 <kart_> Add cx_version to cx_translations table (T190009) [releng]
07:27 <marostegui> Reload dbproxy1002 and dbproxy1007 to get the new config - T189773 [production]
06:20 <marostegui> Deploy schema change on db1091 - T187089 T185128 T153182 [production]
06:13 <marostegui> Stop MySQL on db1091 for kernel and mariadb upgrade [production]
06:13 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1091 for schema change, kernel upgrade and mariadb upgrade (duration: 00m 58s) [production]
02:39 <l10nupdate@tin> scap sync-l10n completed (1.31.0-wmf.25) (duration: 10m 54s) [production]
2018-03-18 §
22:22 <Krinkle> Reloading Zuul to deploy https://gerrit.wikimedia.org/r/420230 [releng]
04:42 <Krinkle> Reloading Zuul to deploy https://gerrit.wikimedia.org/r/417344 [releng]
2018-03-17 §
20:11 <paladox> upgraded gerrit-new yesturday night to include a majour new change https://gerrit-review.googlesource.com/c/gerrit/+/128411 + more fixes [git]
18:41 <elukey> executed apt-get clean on scb1004 to free some space (root partition disk space warning) [production]
03:09 <krinkle@tin> Synchronized docroot/noc/db.php: noc: I410a56431a (duration: 00m 59s) [production]
00:13 <mutante> running puppet on all cache::misc to rename director bromine to webserver_misc_static (T188163) [production]
2018-03-16 §
23:32 <mutante> signing puppet cert for vega.codfw.wmnet, initial puppet run after fresh stretch install (T188163) [production]
22:44 <zhuyifei1999_> suspended process 22825 (BotOrderOfChapters.exe) on tools-bastion-03. Threads continuously going to D-state & R-state. Also sent message via $ write on pts/10 [tools]
20:23 <ottomata> bouncing main -> jumbo mirror makers to apply change-prop topic blacklist [analytics]
18:43 <mutante> creating new ganeti VM vega.codfw.wmnet to be equivalent of bromine, 1G RAM, 30G disk, 1vCPU (T189899) [production]
18:13 <jynus> switching back wikireplica cloud dns to the original config [production]
18:08 <paladox> upgrading gerrit on gerrit-new.wmflabs.org/r/ to latest master version (includes ui tweeks + lots and lots of bug fixes) [git]
17:32 <jynus> reimage dbproxy1010 [production]
16:29 <jynus> updating wikireplica_dns 2/3 [production]
16:22 <moritzm> installing curl security updates [production]
16:09 <marostegui> Stop MySQL on db1020 - T189773 [production]
14:48 <andrewbogott> reset contintcloud quotas as per https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Troubleshooting#incorrect_quota_violations [production]
14:48 <jynus> reimage dbproxy1011 [production]
14:44 <ottomata> restarting eventlogging mysql eventbus consumer to consume from analytics instead of jumbo [analytics]
14:38 <elukey> temporary point pivot to druid1002 as prep step for druid1001's reboot [analytics]
14:37 <elukey> disable druid1001's middlemanager as prep step for reboot [analytics]
14:27 <andrewbogott> restarting nodepool on nodepool1001 [production]
14:25 <elukey> reboot druid1002 for kernel updates [production]
14:24 <elukey> changed superset druid private config from druid1002 to druid1003 [analytics]
14:14 <andrewbogott> restarting rabbitmq on labcontrol1001 [production]
14:13 <Amir1> deleted redis-dispatching-client and redis-dispacthing-repo [wikidata-dev]
13:57 <andrewbogott> stopping nodepool temporarily during changes to nova.conf [production]
13:43 <elukey> disable druid1002's middle manager via API as prep step for reboot [analytics]