201-250 of 10000 results (60ms)
2019-02-04 §
11:36 <arturo> T215154 manually install systemd 239 in tools-sgebastion-06 [tools]
11:04 <moritzm> installing ghostscript security updates [production]
08:48 <jynus> fixing dbstore1002 x1 replication [production]
07:56 <vgutierrez> uploaded certcentral 0.8 to apt.wikimedia.org (stretch) - T209980 T213820 T213301 [production]
2019-02-03 §
20:25 <elukey> powercycle mw1272 - no ssh, no tty available via com2 - DIMM correctable errors + OEM errors registered in getsel [production]
18:56 <elukey> started a tmux session on dbstore1002 to migrate all the tokudb tables of mediawikiwiki to InnoDB - (s3 replication broken) [production]
17:53 <elukey> start all slaves on dbstore1002 (After a crash + recovery) + moved mediawikiwiki.revision_actor_temp to Innodb to unblock s3 slave replication (still broken though) [production]
06:15 <legoktm> deployed https://gerrit.wikimedia.org/r/487627 [releng]
05:44 <legoktm> deployed https://gerrit.wikimedia.org/r/485967 [releng]
04:55 <legoktm@deploy1001> Synchronized wmf-config/extension-list: Remove WikibaseQuality from extensions-list (T208499) (duration: 00m 51s) [production]
04:36 <legoktm> deploying https://gerrit.wikimedia.org/r/487534 [releng]
01:10 <elukey> powercycle mw1299 - can't ssh nor get a tty via console - racadm getsel shows "An OEM diagnostic event occurred." [production]
2019-02-02 §
22:17 <legoktm> legoktm@integration-slave-jessie-1004:/srv/jenkins-workspace/workspace$ sudo rm -rf * [releng]
20:42 <chaomodus> restarted pdfrender on scb1003 [production]
20:41 <chaomodus> restarted pdfrender on scb1004 [production]
20:06 <chaomodus> parsoid was failed on scandium and alerting, the service parsoid-vd was restarted and appears to have come back [production]
05:44 <jforrester@deploy1001> Synchronized php-1.33.0-wmf.14/extensions/VisualEditor/lib/ve/src/ui/dialogs/ve.ui.FindAndReplaceDialog.js: b/src/ui/dialogs/ve.ui.FindAndReplaceDialog.js T214963 Hot-deploy VE fix to stop hitting user pref writes without debounce (duration: 01m 02s) [production]
2019-02-01 §
23:16 <vgutierrez> restart pdfrender on scb1004 [production]
21:57 <ejegg> updated payments-wiki-staging from 7767c7027e to 52a271e681 [production]
21:25 <ejegg> updated payments-wiki-staging to fundraising/REL1_31 branch [production]
09:54 <arturo> moving canary1015-01 VM instance from cloudvirt1024 back to cloudvirt1015 [admin]
07:13 <bawolff_> reset 2FA on wikitech for [[User:Cicalese]] [production]
2019-01-31 §
17:44 <jynus> running alter table on metawiki.revision_actor_temp, trying to fix TokuDB horrible bugs [production]
15:54 <jynus> stop, upgrade and restart db1117 [production]
15:03 <thcipriani> rearm keyholder on deployment-deploy01 [releng]
13:34 <mvolz@deploy1001> scap-helm zotero finished [production]
13:34 <mvolz@deploy1001> scap-helm zotero cluster codfw completed [production]
13:34 <mvolz@deploy1001> scap-helm zotero upgrade production -f zotero-values-codfw.yaml stable/zotero [namespace: zotero, clusters: codfw] [production]
13:31 <mvolz@deploy1001> scap-helm zotero finished [production]
13:31 <mvolz@deploy1001> scap-helm zotero cluster eqiad completed [production]
13:31 <mvolz@deploy1001> scap-helm zotero upgrade production -f zotero-values-eqiad.yaml stable/zotero [namespace: zotero, clusters: eqiad] [production]
13:19 <mvolz@deploy1001> scap-helm zotero finished [production]
13:19 <mvolz@deploy1001> scap-helm zotero cluster staging completed [production]
13:19 <mvolz@deploy1001> scap-helm zotero upgrade staging -f zotero-values-staging.yaml --version=0.0.1 stable/zotero [namespace: zotero, clusters: staging] [production]
13:18 <mvolz@deploy1001> scap-helm zotero upgrade staging -f zotero-values-staging.yaml stable/zotero [namespace: zotero, clusters: staging] [production]
12:54 <jynus> stop, upgrade and restart db2044 [production]
12:44 <arturo> T215012 depooling cloudvirt1015 and migrating all VMs to cloudvirt1024 [admin]
12:44 <arturo> T214012 depooling cloudvirt1015 and migrating all VMs to cloudvirt1024 [admin]
12:12 <jynus> apply new grants to m5-master with replication T214740 [production]
12:07 <arturo> VM instances mediawiki2latex, were stopped briefly due to issue in hypervisor (T215012) [collection-alt-renderer]
12:07 <arturo> VM instances ldfclient-new, were stopped briefly due to issue in hypervisor (T215012) [wikidata-query]
12:07 <arturo> VM instances wikidata-misc, were stopped briefly due to issue in hypervisor (T215012). [wikidata-dev]
12:06 <arturo> VM instances deployment-webperf12,ecmabot, were stopped briefly due to issue in hypervisor (T215012) [webperf]
12:06 <arturo> VM instances encoding02,encoding03, were stopped briefly due to issue in hypervisor (T215012) [video]
12:06 <arturo> VM instances twlight-prod,twlight-staging,twlight-tracker, were stopped briefly due to issue in hypervisor (T215012) [twl]
12:06 <arturo> VM instances packaging, were stopped briefly due to issue in hypervisor (T215012) [thumbor]
12:06 <arturo> VM instances canary1015-01, were stopped briefly due to issue in hypervisor (T215012) [testlabs]
12:06 <arturo> VM instances hafnium,neon,oxygen, were stopped briefly due to issue in hypervisor (T215012) [rcm]
12:05 <arturo> VM instances novaadminmadethis, were stopped briefly due to issue in hypervisor (T215012) [quotatest]
12:05 <arturo> VM instances ores-puppetmaster-01,ores-sentinel-01, were stopped briefly due to issue in hypervisor (T215012) [ores]