4251-4300 of 10000 results (76ms)
2019-06-12 §
11:55 <godog> swift eqiad-prod: put back ms-be1033 - T223518 [production]
10:52 <godog> force-upgrade mtail to 3.0.0~rc24.1-1 on wezen - T225604 [production]
10:36 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: More traffic to db1077 after recovering from a crash (duration: 00m 47s) [production]
10:18 <akosiaris@deploy1001> scap-helm zotero finished [production]
10:18 <akosiaris@deploy1001> scap-helm zotero cluster codfw completed [production]
10:17 <akosiaris@deploy1001> scap-helm zotero cluster eqiad completed [production]
10:17 <akosiaris@deploy1001> scap-helm zotero upgrade --dry-run --debug production stable/zotero [namespace: zotero, clusters: eqiad,codfw] [production]
10:01 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Slowly repool db1077 after a crash (duration: 00m 48s) [production]
09:51 <gehel@cumin2001> END (PASS) - Cookbook sre.elasticsearch.rolling-restart (exit_code=0) [production]
08:59 <hashar> Gracefully stopping Zuul (kill -SIGUSR1) to prepare for the restart of the CI Jenkins T225322 [production]
08:41 <onimisionipe> pool map2003. reimage and setup is complete - T224395 [production]
08:31 <gehel@cumin2001> START - Cookbook sre.elasticsearch.rolling-restart [production]
06:49 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Slowly repool db1077 after a crash (duration: 00m 49s) [production]
2019-06-11 §
19:24 <tzatziki> Removing four (4) files for legal compliance [production]
15:41 <gehel> shutting down elastic1029 for investigation - T214283 [production]
12:54 <godog> swift eqiad-prod: put back ms-be1033 - T223518 [production]
11:52 <gehel@cumin2001> END (PASS) - Cookbook sre.postgresql.postgres-init (exit_code=0) [production]
10:54 <godog> wipe fs on ms-be1033 data partitions - T223518 [production]
09:56 <gehel@cumin2001> START - Cookbook sre.postgresql.postgres-init [production]
09:20 <godog> free up space wrongly allocated onto / with sdc1 umounted on ms-be2018 [production]
08:26 <gehel> repooling maps200[124] [production]
2019-06-10 §
19:39 <thcipriani> restarting jenkins [production]
19:11 <akosiaris> refresh all zotero pods in all clusters [production]
19:11 <akosiaris@deploy1001> scap-helm zotero finished [production]
19:11 <akosiaris@deploy1001> scap-helm zotero cluster staging completed [production]
19:11 <akosiaris@deploy1001> scap-helm zotero upgrade -f zotero-values-staging.yaml staging stable/zotero [namespace: zotero, clusters: staging] [production]
19:11 <akosiaris@deploy1001> scap-helm zotero finished [production]
19:10 <akosiaris@deploy1001> scap-helm zotero cluster eqiad completed [production]
19:10 <akosiaris@deploy1001> scap-helm zotero upgrade -f zotero-values-eqiad.yaml production stable/zotero [namespace: zotero, clusters: eqiad] [production]
19:10 <akosiaris@deploy1001> scap-helm zotero finished [production]
19:10 <akosiaris@deploy1001> scap-helm zotero cluster codfw completed [production]
19:10 <akosiaris@deploy1001> scap-helm zotero upgrade -f zotero-values-codfw.yaml production stable/zotero [namespace: zotero, clusters: codfw] [production]
17:55 <ottomata> rolling restart of AQS service using scap deploy for new mediawiki_history_snaphost [production]
17:55 <otto@deploy1001> Started restart [analytics/aqs/deploy@fc1d232]: (no justification provided) [production]
16:24 <marostegui> Power reset db1077 from the idrac T225391 [production]
13:18 <mvolz@deploy1001> scap-helm citoid finished [production]
13:18 <mvolz@deploy1001> scap-helm citoid cluster codfw completed [production]
13:18 <mvolz@deploy1001> scap-helm citoid upgrade production -f citoid-codfw-values.yaml stable/citoid [namespace: citoid, clusters: codfw] [production]
13:13 <mvolz@deploy1001> scap-helm citoid finished [production]
13:13 <mvolz@deploy1001> scap-helm citoid cluster eqiad completed [production]
13:13 <mvolz@deploy1001> scap-helm citoid upgrade production -f citoid-eqiad-values.yaml stable/citoid [namespace: citoid, clusters: eqiad] [production]
13:04 <mvolz@deploy1001> scap-helm citoid finished [production]
13:04 <mvolz@deploy1001> scap-helm citoid cluster staging completed [production]
13:04 <mvolz@deploy1001> scap-helm citoid upgrade staging -f citoid-staging-values.yaml stable/citoid [namespace: citoid, clusters: staging] [production]
05:37 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1077 - host crashed (duration: 00m 52s) [production]
2019-06-09 §
08:30 <vgutierrez> rebooting lvs4007 after NIC driver crash [production]
2019-06-08 §
11:58 <godog> stop swift processes on ms-be1033 - T223518 [production]
10:46 <reedy@deploy1001> Synchronized wmf-config/throttle.php: T225344 (duration: 00m 51s) [production]
2019-06-07 §
18:56 <herron> performing rolling reboots of logstash codfw frontends for security updates [production]
18:22 <cstone> Update payments-wiki revision changed from c6c7bbf71e to 75abd71cc1 [production]