2023-10-19
§
|
19:58 |
<xcollazo> |
ran "sudo -u hdfs hdfs dfs -cp /user/xcollazo/artifacts/spark-3.3.2-assembly.zip /user/spark/share/lib/" and "sudo -u hdfs hdfs dfs -chmod o+r /user/spark/share/lib/spark-3.3.2-assembly.zip" to bring make Spark 3.3.2 assembly available for other folks. |
[analytics] |
19:54 |
<xcollazo> |
ran "sudo -u hdfs hdfs dfs -rm /user/spark/share/lib/spark-3.1.2-assembly.jar.backup" to remove old spark assembly backup from May 25 2023. |
[analytics] |
19:52 |
<xcollazo> |
ran "$ sudo -u hdfs hdfs dfs -rm /user/spark/share/lib/spark-3.1.2-assembly.jar.bak" to remove old spark assembly backup from Jun 13 2023. |
[analytics] |
15:22 |
<brouberol> |
The kafka service has been stopped on kafka-jumbo100[1-6] - T336044 |
[analytics] |
15:04 |
<brouberol> |
sudo cumin --batch-size 1 --batch-sleep 60 'kafka-jumbo100[1-6].eqiad.wmnet' 'sudo systemctl stop kafka.service' - T336044 |
[analytics] |
15:02 |
<brouberol> |
disabling puppet on kafka-jumbo100[1-6] to make sure kafka isn't resarted - T336044 |
[analytics] |
12:13 |
<brouberol> |
disabling puppet on kafka-jumbo nodes so we can merge https://gerrit.wikimedia.org/r/c/operations/puppet/+/966497 |
[analytics] |
09:42 |
<btullis> |
re-running airflow jobs for missing webrequest data on hadoop-test |
[analytics] |
2023-10-17
§
|
16:17 |
<btullis> |
restarting hadoop-yarn-nodemanager on an-test-worker1001 |
[analytics] |
14:01 |
<tchin> |
deploying airflow analytics |
[analytics] |
13:39 |
<tchin> |
deploying refinery |
[analytics] |
12:56 |
<btullis> |
deploying multiple spark shufflers to the test cluster |
[analytics] |
09:51 |
<btullis> |
re-enabling all previously paused dags |
[analytics] |
09:50 |
<btullis> |
restarting all airflow schedulers after rebooting an-db1001 |
[analytics] |
09:10 |
<btullis> |
pausing both active dags on the analytics_product airflow instance |
[analytics] |
09:09 |
<btullis> |
pausing all 7 active dags on airflow-platform_eng airflow instance |
[analytics] |
09:07 |
<btullis> |
pausing all 3 active dags on airflow-research instance |
[analytics] |
09:07 |
<btullis> |
pausing all 28 active airflow dags on airflow-search instance |
[analytics] |
09:03 |
<btullis> |
pausing all airflow dags on analytics instance |
[analytics] |