4651-4700 of 10000 results (63ms)
2018-12-17 §
08:16 <marostegui> Stop replication on labsdb1011 [production]
08:15 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1123 T86338 T202167 (duration: 00m 45s) [production]
07:58 <marostegui> Enable GTID on db2052 (s5 master) - T211973 [production]
07:55 <AlexZ> rebooted cvnbot18 due to no activity in 3d [cvn]
06:48 <marostegui> Enable GTID on db2035 (s2 master) - T211973 [production]
06:36 <marostegui> Enable GTID on db2034 (x1 master) - T211973 [production]
06:36 <marostegui> Enable GTID on db2034 (x1 master) [production]
06:29 <marostegui> Enable replication consistency options on codfw masters - T211973 [production]
06:28 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1123 T86338 T202167 (duration: 00m 46s) [production]
06:28 <marostegui> Depool labsdb1010 - T86338 [production]
04:22 <tstarling@deploy1001> Synchronized php-1.33.0-wmf.8/extensions/AbuseFilter/maintenance/normalizeThrottleParameters.php: gerrit 479998 for testing of normalizeThrottleParameters.php (duration: 00m 44s) [production]
04:20 <tstarling@deploy1001> Synchronized php-1.33.0-wmf.8/extensions/AbuseFilter/includes/AbuseFilter.php: gerrit 479998 for testing of normalizeThrottleParameters.php (duration: 00m 59s) [production]
2018-12-16 §
09:52 <elukey> mask + reset-failed kafkatee default instance on sulfur (kafkatee-webrequest works fine) [production]
07:30 <marostegui> Reboot db1115 after OOM [production]
07:28 <marostegui> Stop MySQL on db1115 so tendril can get back to work [production]
2018-12-15 §
17:08 <Nettrom> shut down and deleted suggestbot-prod [suggestbot]
13:03 <volans|off> restarted pdfrender on scb1004 [production]
09:22 <elukey> mask + reset-failed kafkatee default instance on weblog1001 [production]
07:36 <Nettrom> completed installation on suggestbot-01, instated crontab [suggestbot]
07:35 <Nettrom> installed libenchant-dev on suggestbot-01 [suggestbot]
07:34 <Nettrom> imported backup of SuggestBot’s code on suggestbot-01 [suggestbot]
07:04 <Nettrom> imported backup of SuggestBot’s database on suggestbot-01 [suggestbot]
2018-12-14 §
23:59 <mutante> LDAP: added aezell to wmf group (T211945) for grafana access [production]
22:00 <XioNoX> increase accepted-prefix-limit for HE to 200000 [production]
21:28 <andrewbogott> moving VMs to eqiad1 [openstack]
20:45 <ottomata> re-refining all hive EventLogging tables since 2018-11-29T17:00:00. - T211833 [analytics]
20:38 <mutante> sulfur systemctl restart nagios-nrpe-server [production]
20:35 <ottomata> removing EventLogging Hive _REFINED flag files since 2018-11-29T17:00:00 to allow for re-refinement of data - 2018-11-29T17:00:00 [analytics]
20:32 <andrew@deploy1001> Finished deploy [horizon/deploy@1a830b9]: Rolling out fix for T177855 (duration: 03m 17s) [production]
20:29 <andrew@deploy1001> Started deploy [horizon/deploy@1a830b9]: Rolling out fix for T177855 [production]
20:13 <otto@deploy1001> Finished deploy [analytics/refinery@ef1f7c6]: deploying refinery-source 0.0.82 with fix for T211833 (duration: 06m 04s) [production]
20:07 <otto@deploy1001> Started deploy [analytics/refinery@ef1f7c6]: deploying refinery-source 0.0.82 with fix for T211833 [production]
20:07 <otto@deploy1001> deploy aborted: (no justification provided) (duration: 00m 00s) [production]
20:07 <otto@deploy1001> Started deploy [analytics/refinery@ef1f7c6]: (no justification provided) [production]
19:50 <ottomata> staring refinery release deploy process for refinery 0.0.82 to fix T211833 [analytics]
18:39 <mutante> - ran deploy-wikistats deploy, copied .xml files, migrationg to new region should be all done now [wikistats]
18:25 <mutante> - replaced wikistats-kraken (eqiad) with wikistats-greyhound (eqiad-r), new stretch image, copied mysql data over, deleted and recreated web proxy [wikistats]
17:22 <cmjohnson1> mw1272 down for h/w troubleshooting [production]
16:41 <andrewbogott> deleting project as per https://phabricator.wikimedia.org/T204519 [mwfileimport]
16:19 <Krenair> T204500 Powering off the remaining trusty and main deployment instance, deployment-zotero01, which is no longer in use anyway [releng]
14:43 <andrewbogott> moving project to eqiad1-r [butterfly]
14:41 <andrewbogott> deleting 'wigi' VM at Maximilian's request. [wikidumpparse]
14:38 <marostegui> Enable GTID on db2039 (s6 codfw master) - T211973 [production]
14:24 <andrewbogott> migrating project to eqiad1-r [wikistats]
14:17 <andrewbogott> migrating project to eqiad1-r [queryrapi]
14:03 <marostegui> Compare ruwiki.revision between db2039 (s6 master) and db1085 - T211973 [production]
13:59 <marostegui> Enable notifications for db1095 (s3 lag check)- T211973 [production]
13:57 <marostegui> Enable notifications for db2068 (s7 lag check)- T211973 [production]
13:47 <dcausse> elastic@codfw copying index data from the main cluster to psi & omega (test disk usage & import speed) [production]
13:42 <marostegui> Enable GTID on db1124:3318 - T211973 [production]