551-600 of 2137 results (15ms)
2018-10-15 §
14:47 <mforns> Finished refinery-source deployment [analytics]
14:19 <mforns> Started refinery-source deployment [analytics]
14:05 <elukey> swapped cobalt's ip with gerrit.wikimedia.org's one in analytics-in(4|6) firewall filters on the eqiad routers for https://phabricator.wikimedia.org/T206331#4666622. This should not cause git pulls to fail but let me know in case it does. [analytics]
2018-10-14 §
09:15 <elukey> restart yarn resource manager on an-coord1002 (failover happened due to jvm issues) [analytics]
09:15 <elukey> restart apps-session-metrics with spark 2.3.1 oozie libs (modified the coordinator.properties file manually on disk) [analytics]
2018-10-12 §
07:32 <elukey> cleaned up all september files from eventlog1002's srv el archive to free some space (disk alerts) [analytics]
2018-10-11 §
14:20 <elukey> reboot eventlog1002 for kernel upgrades [analytics]
2018-10-10 §
19:27 <joal> Restart webrequest-load oozie bundle [analytics]
18:23 <joal> kill Webrequest-load bundle [analytics]
18:04 <joal> Kill webrequest-load-coord-upload [analytics]
07:23 <elukey> add ipv6 mapped addresses (and DNS PTRs) to analytics-tools* [analytics]
07:23 <joal> Full restart of browser-general oozie job [analytics]
07:19 <joal> patch mediacount-archive job in prod [analytics]
07:16 <joal> Full restart of mediacount-archive oozie job [analytics]
05:54 <elukey> re-run failed mediacounts and browser-general coordinators with hive-site -> hdfs://analytics-hadoop/user/hive/hive-site.xml [analytics]
2018-10-09 §
18:24 <ottomata> adding Accept header to all varnishkafka generated webrequest logs [analytics]
15:10 <joal> restart Mediawiki-history-reduced [analytics]
15:08 <joal> restart wikidata-coeditors oozie job [analytics]
15:08 <joal> restart wikidata-specialentites oozie job [analytics]
15:00 <joal> restart wikidata-article-placeholder oozie job [analytics]
14:57 <joal> restart mediawiki-history denormalize oozie job [analytics]
14:56 <joal> Restart check_denormalize oozie job [analytics]
14:53 <joal> Restart clickstream oozie job to pick new spark-lib [analytics]
13:56 <ottomata> bouncing oozie server on an-coord1001 [analytics]
13:46 <joal> Restarting oozie-api job [analytics]
13:36 <joal> fully restart projectview_geo oozier job [analytics]
13:26 <joal> Full restart of aqs oozie job [analytics]
13:25 <joal> full restart of projectview_hourly [analytics]
13:14 <joal> rerun failed aqs-hourl jobs [analytics]
12:48 <elukey> re-run all the failed projectview-hourly-coord and aqs-hourly-coord workflows (restarting them via hue) [analytics]
12:47 <elukey> re-run apis-wf-2018-10-9-8 [analytics]
10:01 <joal> Restart failed oozie jobs (webrequest, virtual-pageviews, mwh-reduced) [analytics]
07:14 <elukey> stopped all crons on analytics1003 as prep step for migration to an-coord1001 [analytics]
2018-10-08 §
16:28 <elukey> restart eventlogging on eventlog1002 for python security upgrades [analytics]
10:26 <elukey> swapped db settings from analytics1003 to an-coord1001 on both Druid clusters (restarted coordinators and overlords) [analytics]
07:35 <joal> Manually run download-project-namespace-map with proxy [analytics]
2018-10-06 §
18:10 <elukey> restart Yarn Resource Manager on an-master1002 to force an-master1001 to take the active role back (failed over due to a zk conn issue) [analytics]
2018-10-05 §
10:32 <elukey> piwik/matomo out of maintenance [analytics]
10:17 <elukey> set piwik/matomo in maintenance mode on matomo1001 [analytics]
2018-10-04 §
20:33 <mforns> Finished deployment of refinery [analytics]
19:52 <mforns> Started deployment of refinery [analytics]
19:50 <mforns> Finished deployment of refinery-source [analytics]
19:22 <mforns> Started deployment of refinery-source [analytics]
17:20 <elukey> bounce druid-brokers on druid100[4-6] after network maintenance [analytics]
2018-10-01 §
12:56 <fdans> reverting to last version of wikistats [analytics]
2018-09-27 §
06:44 <elukey> rolling restart of Druid coordinators and historicals on the Druid public cluster to pick up new Hadoop masters (one at the time, very gently) [analytics]
2018-09-26 §
20:39 <elukey> rolling restart of all the druid historicals on Druid private/analytics [analytics]
20:00 <ottomata> rolling restart of druid coordinators to hopefully pick up hadoop master config change [analytics]
17:49 <joal> Deploy AQS from scap [analytics]
08:22 <elukey> start mysql consumers on eventlog1002 after maintenance [analytics]