6001-6050 of 10000 results (99ms)
2019-10-14 §
08:51 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1074 and db2126 after changing sanitarium to replicate from db1074 T231638', diff saved to https://phabricator.wikimedia.org/P9322 and previous config saved to /var/cache/conftool/dbconfig/20191014-085143-marostegui.json [production]
08:46 <mobrovac> restbase drop metadata keyspaces from cassandra - T235173 [production]
07:54 <marostegui> Stop db1074 and db2126 in sync to change sanitarium's master for s2 - T231638 [production]
07:49 <mobrovac@deploy1001> Finished deploy [restbase/deploy@4d469a1] (dev-cluster): Remove VE logging and stop using storage for /page/metadata (duration: 03m 58s) [production]
07:45 <mobrovac@deploy1001> Started deploy [restbase/deploy@4d469a1] (dev-cluster): Remove VE logging and stop using storage for /page/metadata [production]
07:41 <mobrovac@deploy1001> Finished deploy [restbase/deploy@e0d071f]: Remove VE logging and stop using storage for /page/metadata - T234928 T235173 (duration: 13m 37s) [production]
07:33 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1074 and db2126 to change sanitarium to replicate from db1074 T231638', diff saved to https://phabricator.wikimedia.org/P9320 and previous config saved to /var/cache/conftool/dbconfig/20191014-073319-marostegui.json [production]
07:28 <mobrovac@deploy1001> Started deploy [restbase/deploy@e0d071f]: Remove VE logging and stop using storage for /page/metadata - T234928 T235173 [production]
07:27 <mobrovac@deploy1001> Finished deploy [changeprop/deploy@c25a1c2]: Do not pre-generate /page/metadata - T235173 (duration: 01m 25s) [production]
07:26 <mobrovac@deploy1001> Started deploy [changeprop/deploy@c25a1c2]: Do not pre-generate /page/metadata - T235173 [production]
07:21 <marostegui@cumin1001> dbctl commit (dc=all): 'Remove db2068 from config - T235399', diff saved to https://phabricator.wikimedia.org/P9319 and previous config saved to /var/cache/conftool/dbconfig/20191014-072100-marostegui.json [production]
07:16 <marostegui> Stop MySQL on labsdb1009 for on-site maintenance - T233273 [production]
07:01 <marostegui@cumin1001> END (FAIL) - Cookbook sre.hosts.decommission (exit_code=1) [production]
07:01 <marostegui@cumin1001> START - Cookbook sre.hosts.decommission [production]
06:03 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Remove db2068 from config T235399 (duration: 00m 51s) [production]
06:02 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Remove db2068 from config T235399 (duration: 00m 53s) [production]
05:47 <marostegui> Remove db2068 from tendril and zarcillo T235399 [production]
04:56 <marostegui> Depool labsdb1009 for on-site maintenance - T233273 [production]
04:56 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1101:3317 for schema change T233625', diff saved to https://phabricator.wikimedia.org/P9318 and previous config saved to /var/cache/conftool/dbconfig/20191014-045629-marostegui.json [production]
02:52 <bd808> Reduced report job frequency to once per hour [tools.os-deprecation]
02:50 <bd808> Killed stuck update job running since 2019-10-07 [tools.os-deprecation]
2019-10-13 §
22:45 <legoktm> rebooting upgrader-05 because of DNS failures when run in docker [library-upgrader]
21:52 <wm-bot> <lokal-profil> move dologmsg from /usr/bin/ to /usr/local/bin/ [finding-glams]
21:19 <wm-bot> <lokal-profil> Copy deploy_message_from_git_log.py from heritage repo (T235301) [finding-glams]
21:06 <wm-bot> <lokal-profil> Adding dologmsg to /usr/bin [finding-glams]
00:52 <krinkle@deploy1001> Synchronized wmf-config/CommonSettings.php: ec77b1b515940c73 (duration: 00m 55s) [production]
2019-10-12 §
23:21 <krinkle@deploy1001> Synchronized wmf-config/profiler.php: bfa8bb69c1f, T231564 (duration: 00m 51s) [production]
22:17 <wm-bot> <jeanfred> Deleted old GridEngine jobs from Oct 7 (jstop update_monuments jstop categorize_images) [tools.heritage]
22:17 <wm-bot> <jeanfred> Deleted old GridEngine jobs from Oct 7 (jstop update_monuments jstop categorize_images) [tools.wikiloves]
22:17 <wm-bot> <jeanfred> Deleted old GridEngine jobs from Oct 7 ( ) [tools.wikiloves]
21:07 <krinkle@deploy1001> Synchronized php-1.35.0-wmf.1/includes/resourceloader/ResourceLoaderStartUpModule.php: 8c6baeae2 (duration: 00m 53s) [production]
20:57 <Urbanecm> Reset user email of User:Gardini (T235318) [production]
18:38 <_joe_> deleting zotero pods with excessive memory usage in eqiad [production]
16:43 <Reedy> Reloading Zuul to deploy https://gerrit.wikimedia.org/r/542690 [releng]
16:16 <reedy@deploy1001> Synchronized php-1.35.0-wmf.1/includes/api/ApiQueryBase.php: T235334 (duration: 00m 51s) [production]
16:15 <reedy@deploy1001> Synchronized php-1.35.0-wmf.1/includes/api/ApiQueryBacklinksprop.php: T235334 (duration: 00m 56s) [production]
15:43 <valhallasw`cloud> qdel'ed wb2-grrrrit and restarted with ~/py35-stretch/bin/python manage.py start_jobs [tools.wikibugs]
04:37 <krinkle@deploy1001> Synchronized wmf-config/profiler.php: 29d846938c898dd (duration: 00m 57s) [production]
2019-10-11 §
21:52 <jeh> create toolforge.org DNS zone in tools project T234617 [openstack]
21:16 <jeh> grant tools-dns-manager designateadmin role in tools project T235304 [openstack]
19:52 <bstorm_> restarted docker on tools-docker-builder after phamhi noticed the daemon had a routing issue (blank iptables) [tools]
18:24 <Lucas_WMDE> wikidata-shex deleted instance (T234313) [wikidata-dev]
17:57 <Krinkle> Click "Disable Publishing" on extension-Elastica Phab mirror, T235233, T143162 [releng]
17:30 <wikibugs> Updated channels.yaml to: f51498e134a76fe72b944e1db7c5f6f38dd16f44 Update tags for #mediawiki-i18n [tools.wikibugs]
16:45 <James_F> Zuul: Add dependency "BlueSpiceExtendedSearch" to BlueSpiceStatistics [releng]
16:03 <paladox> ssh -p 29418 gerrit.wikimedia.org gerrit create-project cloud.git --permissions-only - T235279 [releng]
15:59 <paladox> ssh -p 29418 gerrit.wikimedia.org gerrit create-project cloud.git --permissions-only --description="Container for all other cloud/* projects for permissions, etc." - T235279 [releng]
15:39 <AndyRussG> updated fruec from 18d89675d0 to 1e6a6ee2de [production]
13:57 <moritzm> rebooting cloudbackup2001 [production]
13:57 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]