6151-6200 of 10000 results (47ms)
2017-04-24 §
09:15 <marostegui> Stop MYSQL on db1062 to backup its mysql - T163665 [production]
09:14 <jynus> dropping ptwikimedia from es1012,es1016,es1018,es2011,es2012,es2013, T126832 [production]
09:11 <jynus> dropping ptwikimedia from es3 T126832 [production]
09:08 <jynus> dropping ptwikimedia from es2 T126832 [production]
09:04 <jynus> dropping ptwikimedia from x1 T126832 [production]
08:55 <jynus> dropping ptwikimedia from s3 T126832 [production]
08:02 <marostegui> Deploy alter table enwiki.revision on db1095 (sanitarium2) - T132416 [production]
07:34 <marostegui@naos> Synchronized wmf-config/db-eqiad.php: Repool db1080 and db1067 (duration: 01m 18s) [production]
06:23 <marostegui> Deploy alter table enwiki.revision db1052 (eqiad master) - T132416 [production]
06:12 <marostegui> Deploy alter table on wikidatawiki.wb_terms on db1087 - https://phabricator.wikimedia.org/T162539 https://phabricator.wikimedia.org/T163548 [production]
06:12 <marostegui@naos> Synchronized wmf-config/db-eqiad.php: Repool db1092, depoll db1087 - T162539 T163548 (duration: 02m 19s) [production]
2017-04-23 §
19:13 <ema> cp2020: restart varnish-be [production]
17:49 <jynus> disabling puppet on db2062 and upgrading MariaDB package to 10.1 T116557 [production]
03:11 <andrewbogott> removing files in /srv/deployment/ocg/postmortem on ocg1003, another case of T162780 [production]
2017-04-22 §
13:41 <ema@neodymium> conftool action : set/pooled=no; selector: name=cp2024.codfw.wmnet,service=varnish-be [production]
07:53 <jynus> restarting es2019.codfw.wmnet after upgrade [production]
07:43 <jynus> powercycling es2019.codfw.wmnet, unresponsive [production]
07:21 <jynus@naos> Synchronized wmf-config/db-codfw.php: Depool es2019 (duration: 02m 16s) [production]
03:21 <bblack@neodymium> conftool action : set/pooled=yes; selector: name=cp2024.codfw.wmnet,service=varnish-be [production]
02:56 <bblack@neodymium> conftool action : set/pooled=no; selector: name=cp2024.codfw.wmnet,service=varnish-be [production]
02:18 <bblack@neodymium> conftool action : set/pooled=yes; selector: name=cp2002.codfw.wmnet,service=varnish-be [production]
00:34 <bblack@neodymium> conftool action : set/pooled=no; selector: name=cp2002.codfw.wmnet,service=varnish-be [production]
2017-04-21 §
23:52 <bblack@neodymium> conftool action : set/pooled=yes; selector: name=cp2026.codfw.wmnet,service=varnish-be [production]
22:49 <bblack@neodymium> conftool action : set/pooled=no; selector: name=cp2026.codfw.wmnet,service=varnish-be [production]
15:06 <marostegui@naos> Synchronized wmf-config/db-codfw.php: Increase weight db2071 (duration: 01m 17s) [production]
14:32 <marostegui> Analyze revision, logging and page table on s1 db1067 - https://phabricator.wikimedia.org/T116557 [production]
14:26 <ema> ban objects with CT < 1024 on codfw cache_upload T145661 [production]
14:00 <moritzm> installing postgresql bugfix update from jessie point release on labsdb1004 [production]
13:35 <marostegui> Deploy alter table on wikidatawiki.wb_terms on db1092 - T162539 T163548 [production]
13:20 <marostegui@naos> Synchronized wmf-config/db-eqiad.php: Depool db1092 - T162539 T163548 (duration: 01m 18s) [production]
12:51 <akosiaris> reboot puppetmaster1002 for kernel upgrade [production]
12:07 <marostegui> Analyze revision, logging and page table on s1 db1080 - T116557 [production]
12:07 <marostegui@naos> Synchronized wmf-config/db-eqiad.php: Update db1080 depool reason (duration: 01m 18s) [production]
10:35 <marostegui@naos> Synchronized wmf-config/db-eqiad.php: Repool db1071 - T163109 (duration: 01m 20s) [production]
09:20 <moritzm> rebooting etherpad1001 (running etherpad.wikimedia.org) for update to Linux 4.9 [production]
09:10 <jynus> stopping and upgrading/reconfiguring db2062 (depooled) T116557 [production]
08:49 <jynus@naos> Synchronized wmf-config/db-codfw.php: Depool db2062 (duration: 01m 20s) [production]
08:32 <akosiaris> looking at tcpircbot (logmsgbot) problems at tegmen [production]
08:20 <elukey> rolling restart of aqs (nodejs) on aqs* to pick up upgrades [production]
08:01 <moritzm> rolling restart of hhvm on application servers in eqiad to pick up ICU security update [production]
07:47 <marostegui> Stop MySQL on db1071 and db1063 to reclone db1063 - T163109 [production]
07:43 <moritzm> installing further icu security updates [production]
06:21 <marostegui> Restart MySQL on db1065 for maintenance - T163351 [production]
06:09 <marostegui> Deploy alter table enwiki.revision db1067 - T132416 [production]
2017-04-20 §
22:28 <twentyafterfour> enable rate limiting in phabricator [production]
22:17 <paravoid> setting tw_reuse to 1 on dbproxy1003 [production]
21:47 <twentyafterfour> started phd on iridium [production]
21:31 <twentyafterfour> stopped phd on iridium to reduce load on the database [production]
19:26 <Amir1> deploy finished [production]
19:24 <Amir1> start of ladsgroup@naos:/srv/mediawiki-staging/php-1.29.0-wmf.20$ scap sync-file php-1.29.0-wmf.20/extensions/ORES/includes/Hooks.php '[[gerrit:349271|Disable ORES in Recentchangeslinked]] (T163063)' [production]