551-600 of 10000 results (74ms)
2018-10-10 §
02:55 <ejegg> updated payments-wiki from 1472604b6e to 7fb1aae963 [production]
00:19 <krinkle@deploy1001> Synchronized php-1.32.0-wmf.24/includes/utils/UIDGenerator.php: T94522 - I2a0c51bea58 (duration: 00m 56s) [production]
00:15 <krinkle@deploy1001> sync-file aborted: T205567 - I75f1eb6dc2cb (duration: 00m 01s) [production]
00:14 <krinkle@deploy1001> Synchronized php-1.32.0-wmf.24/tests/phpunit/includes/utils/: T94522 - I2a0c51bea58 (duration: 01m 02s) [production]
2018-10-09 §
22:58 <SMalyshev> repooled wdqs2003 [production]
22:26 <shdubsh> repairing /dev/sdl1 on ms-be2040 - T199198 [production]
21:52 <bblack> cp1085: varnish backend restart for mbox lag [production]
21:50 <mutante> releases1001 - restarted jenkins (it went from 200 -> 503 -> 403) curl localhost:8080 works again after restart, icinga check still getting 403 now [production]
21:41 <ejegg|food> updated fundraising CiviCRM from 7a0d14015e to 1165e7ed79 [production]
20:08 <mutante> repair /dev/sdg1 on ms-be2041 - T199198 [production]
19:37 <XioNoX> disable igmp-snooping on asw2-c-eqiad - T201039 [production]
19:25 <XioNoX> disable igmp-snooping on asw2-b-eqiad - T201039 [production]
19:20 <XioNoX> bounce igmp-snooping on asw2-b-eqiad [production]
18:24 <ottomata> adding Accept header to all varnishkafka generated webrequest logs [production]
17:21 <SMalyshev> depooled wdq23 again, sigh [production]
13:54 <moritzm> rebooting prometheus1004 for kernel security update [production]
13:41 <moritzm> rebooting prometheus1003 for kernel security update [production]
13:28 <moritzm> rebooting prometheus2004 for kernel security update [production]
13:13 <moritzm> rebooting prometheus2003 for kernel security update [production]
12:54 <gehel> silencing wdqs-public lag alerts (service still functional, and SLO unclear) - T199228 [production]
12:45 <moritzm> installing imagemagick security updates [production]
11:47 <END> (ERROR) - Cookbook sre.switchdc.services.00-reduce-ttl-and-sleep (exit_code=2) (volans@neodymium) [production]
11:46 <START> - Cookbook sre.switchdc.services.00-reduce-ttl-and-sleep (volans@neodymium) [production]
11:45 <akosiaris> dry-run services switchover from codfw to eqiad in preparation for Thursday [production]
11:37 <END> (ERROR) - Cookbook sre.switchdc.services.00-reduce-ttl-and-sleep (exit_code=2) (volans@neodymium) [production]
11:37 <START> - Cookbook sre.switchdc.services.00-reduce-ttl-and-sleep (volans@neodymium) [production]
11:14 <volans> live-test of the inverted switchdc (eqiad->codfw) completed, all good - T203777 [production]
11:14 <END> (PASS) - Cookbook sre.switchdc.mediawiki.08-update-tendril (exit_code=0) (volans@neodymium) [production]
11:13 <START> - Cookbook sre.switchdc.mediawiki.08-update-tendril (volans@neodymium) [production]
11:12 <END> (PASS) - Cookbook sre.switchdc.mediawiki.08-start-maintenance (exit_code=0) (volans@neodymium) [production]
11:11 <START> - Cookbook sre.switchdc.mediawiki.08-start-maintenance (volans@neodymium) [production]
11:11 <END> (PASS) - Cookbook sre.switchdc.mediawiki.08-restore-ttl (exit_code=0) (volans@neodymium) [production]
11:11 <START> - Cookbook sre.switchdc.mediawiki.08-restore-ttl (volans@neodymium) [production]
11:11 <END> (PASS) - Cookbook sre.switchdc.mediawiki.07-set-readwrite (exit_code=0) (volans@neodymium) [production]
11:11 <[DRY-RUN]> MediaWiki read-only period ends at: 2018-10-09 11:11:05.042622 (volans@neodymium) [production]
11:11 <START> - Cookbook sre.switchdc.mediawiki.07-set-readwrite (volans@neodymium) [production]
11:08 <END> (PASS) - Cookbook sre.switchdc.mediawiki.06-set-db-readwrite (exit_code=0) (volans@neodymium) [production]
11:08 <START> - Cookbook sre.switchdc.mediawiki.06-set-db-readwrite (volans@neodymium) [production]
11:07 <END> (PASS) - Cookbook sre.switchdc.mediawiki.05-invert-redis-sessions (exit_code=0) (volans@neodymium) [production]
11:07 <START> - Cookbook sre.switchdc.mediawiki.05-invert-redis-sessions (volans@neodymium) [production]
11:06 <END> (PASS) - Cookbook sre.switchdc.mediawiki.04-switch-traffic (exit_code=0) (volans@neodymium) [production]
11:04 <START> - Cookbook sre.switchdc.mediawiki.04-switch-traffic (volans@neodymium) [production]
11:03 <END> (PASS) - Cookbook sre.switchdc.mediawiki.04-switch-mediawiki (exit_code=0) (volans@neodymium) [production]
11:03 <START> - Cookbook sre.switchdc.mediawiki.04-switch-mediawiki (volans@neodymium) [production]
11:00 <END> (PASS) - Cookbook sre.switchdc.mediawiki.03-set-db-readonly (exit_code=0) (volans@neodymium) [production]
10:59 <START> - Cookbook sre.switchdc.mediawiki.03-set-db-readonly (volans@neodymium) [production]
10:56 <END> (PASS) - Cookbook sre.switchdc.mediawiki.02-set-readonly (exit_code=0) (volans@neodymium) [production]
10:56 <[DRY-RUN]> MediaWiki read-only period starts at: 2018-10-09 10:56:12.213026 (volans@neodymium) [production]
10:56 <START> - Cookbook sre.switchdc.mediawiki.02-set-readonly (volans@neodymium) [production]
10:53 <END> (PASS) - Cookbook sre.switchdc.mediawiki.01-stop-maintenance (exit_code=0) (volans@neodymium) [production]