851-900 of 10000 results (56ms)
2019-06-12 §
08:31 <gehel@cumin2001> START - Cookbook sre.elasticsearch.rolling-restart [production]
06:49 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Slowly repool db1077 after a crash (duration: 00m 49s) [production]
2019-06-11 §
19:24 <tzatziki> Removing four (4) files for legal compliance [production]
15:41 <gehel> shutting down elastic1029 for investigation - T214283 [production]
12:54 <godog> swift eqiad-prod: put back ms-be1033 - T223518 [production]
11:52 <gehel@cumin2001> END (PASS) - Cookbook sre.postgresql.postgres-init (exit_code=0) [production]
10:54 <godog> wipe fs on ms-be1033 data partitions - T223518 [production]
09:56 <gehel@cumin2001> START - Cookbook sre.postgresql.postgres-init [production]
09:20 <godog> free up space wrongly allocated onto / with sdc1 umounted on ms-be2018 [production]
08:26 <gehel> repooling maps200[124] [production]
2019-06-10 §
19:39 <thcipriani> restarting jenkins [production]
19:11 <akosiaris> refresh all zotero pods in all clusters [production]
19:11 <akosiaris@deploy1001> scap-helm zotero finished [production]
19:11 <akosiaris@deploy1001> scap-helm zotero cluster staging completed [production]
19:11 <akosiaris@deploy1001> scap-helm zotero upgrade -f zotero-values-staging.yaml staging stable/zotero [namespace: zotero, clusters: staging] [production]
19:11 <akosiaris@deploy1001> scap-helm zotero finished [production]
19:10 <akosiaris@deploy1001> scap-helm zotero cluster eqiad completed [production]
19:10 <akosiaris@deploy1001> scap-helm zotero upgrade -f zotero-values-eqiad.yaml production stable/zotero [namespace: zotero, clusters: eqiad] [production]
19:10 <akosiaris@deploy1001> scap-helm zotero finished [production]
19:10 <akosiaris@deploy1001> scap-helm zotero cluster codfw completed [production]
19:10 <akosiaris@deploy1001> scap-helm zotero upgrade -f zotero-values-codfw.yaml production stable/zotero [namespace: zotero, clusters: codfw] [production]
17:55 <ottomata> rolling restart of AQS service using scap deploy for new mediawiki_history_snaphost [production]
17:55 <otto@deploy1001> Started restart [analytics/aqs/deploy@fc1d232]: (no justification provided) [production]
16:24 <marostegui> Power reset db1077 from the idrac T225391 [production]
13:18 <mvolz@deploy1001> scap-helm citoid finished [production]
13:18 <mvolz@deploy1001> scap-helm citoid cluster codfw completed [production]
13:18 <mvolz@deploy1001> scap-helm citoid upgrade production -f citoid-codfw-values.yaml stable/citoid [namespace: citoid, clusters: codfw] [production]
13:13 <mvolz@deploy1001> scap-helm citoid finished [production]
13:13 <mvolz@deploy1001> scap-helm citoid cluster eqiad completed [production]
13:13 <mvolz@deploy1001> scap-helm citoid upgrade production -f citoid-eqiad-values.yaml stable/citoid [namespace: citoid, clusters: eqiad] [production]
13:04 <mvolz@deploy1001> scap-helm citoid finished [production]
13:04 <mvolz@deploy1001> scap-helm citoid cluster staging completed [production]
13:04 <mvolz@deploy1001> scap-helm citoid upgrade staging -f citoid-staging-values.yaml stable/citoid [namespace: citoid, clusters: staging] [production]
05:37 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1077 - host crashed (duration: 00m 52s) [production]
2019-06-09 §
08:30 <vgutierrez> rebooting lvs4007 after NIC driver crash [production]
2019-06-08 §
11:58 <godog> stop swift processes on ms-be1033 - T223518 [production]
10:46 <reedy@deploy1001> Synchronized wmf-config/throttle.php: T225344 (duration: 00m 51s) [production]
2019-06-07 §
18:56 <herron> performing rolling reboots of logstash codfw frontends for security updates [production]
18:22 <cstone> Update payments-wiki revision changed from c6c7bbf71e to 75abd71cc1 [production]
15:34 <godog> bounce rsyslog on wezen - T199406 [production]
15:09 <elukey> restart thorium for kernel upgrades [production]
14:00 <ema> pool cp3039 w/ ATS backend T222937 [production]
13:15 <ema> depool cp3039 and reimage as upload_ats T222937 [production]
13:04 <arturo> aborrero@cumin1001:~ $ sudo cumin "P{R:Systemd::Timer::Job}" "puppet agent --enable && run-puppet-agent" (patch already merged) [production]
13:03 <arturo> aborrero@cumin1001:~$ sudo cumin "P{R:Systemd::Timer::Job}" "puppet agent --disable 'arturo merging systemd timer nrpe change'" (19 hosts affected) merging: https://gerrit.wikimedia.org/r/c/operations/puppet/+/514988 [production]
11:45 <ema> pool cp3043 w/ ATS backend T222937 [production]
10:51 <jbond42> upload libcpp-hocon0.1.6_0.1.6-1~bpo9+1_amd64.deb to wikimedia-stretch component/facter3 [production]
10:45 <jbond42> upload libleatherman-data_1.4.0+dfsg-1\~bpo9+1_all.deb to wikimedia-stretch component/facter3 [production]
10:43 <ema> depool cp3043 and reimage as upload_ats T222937 [production]
10:09 <_joe_> restarting php-fpm on the codfw hosts to pick up the recent changes in opcache [production]