1-50 of 141 results (14ms)
2019-07-11 §
22:07 <bd808> Ran `sudo systemctl stop designate_floating_ip_ptr_records_updater.service` on cloudcontrol1003 [admin]
22:01 <bd808> `sudo apt-get install python2.7-dbg` on cloudcontrol1003 to debug hung python process [admin]
21:48 <bd808> Ran `sudo systemctl stop designate_floating_ip_ptr_records_updater.service` on cloudcontrol1004 [admin]
2019-06-25 §
16:05 <bstorm_> updated python3.4 to update4 wherever it was installed on Jessie VMs to prevent issues with broken update3. [admin]
14:56 <bstorm_> Updated python 3.4 on the labs-puppetmaster server [admin]
2019-06-03 §
15:55 <arturo> T221769 rebooting cloudservices1003 after bootstrapping is apparently completed [admin]
2019-05-28 §
21:42 <bstorm_> unmounting labstore1003-scratch on all cloud clients [admin]
18:14 <bstorm_> T209527 switched mounts from labstore1003 to cloudstore1008 for scratch [admin]
2019-05-20 §
17:25 <arturo> T223923 dropped compat-network config from /etc/network/interfaces in eqiad1/codfw1dev neutron nodes [admin]
17:22 <arturo> T223923 dropped br-compat bridges and vlan interfaces (1102 and 2102) in eqiad1/codfw1dev neutron nodes [admin]
17:07 <arturo> T223923 dropped compat-network configuration from the neutron database in eqiad1 [admin]
16:55 <arturo> T223923 dropped compat-network configuration from the neutron database in codfw1dev [admin]
2019-05-15 §
17:00 <andrewbogott> touching /root/firstboot_done on all VMs that cumin can reach. This will prevent firstboot.sh from running a second time if/when any of these are rebooted. T223370 [admin]
2019-04-26 §
15:51 <arturo> andrew updated dns servers for the cloud-instances2-b-eqiad subnet in neutron: 208.80.154.143 and 208.80.154.24 [admin]
2019-04-25 §
11:14 <arturo> T221760 increased size of conntrack table [admin]
2019-04-24 §
12:54 <arturo> T220051 puppet broken in every VM in Cloud VPS, fixing right now [admin]
2019-04-22 §
11:14 <arturo> create by hand /var/cache/labsaliaser/labs-ip-aliases.json in cloudservices2002-dev (T218575) [admin]
2019-04-16 §
22:55 <bd808> cloudcontrol2003-dev: added `exit 0` to /etc/cron.hourly/keystone to stop cron spam on partially configured cluster [admin]
12:08 <arturo> rebooting cloudvirt200[123]-dev because deep changes in config [admin]
11:27 <arturo> T219626 add DB grants for neutron and glnace to clouddb2001-dev (codfw1dev) [admin]
10:37 <arturo> T219626 replace 208.80.153.75 with 208.80.153.59 in the clouddb2001-dev database (codfw1dev deployment) [admin]
10:29 <arturo> T219626 replace labtestcontrol2003 with cloudcontrol2001-dev in the clouddb2001-dev database (codfw1dev deployment) [admin]
2019-04-15 §
13:08 <arturo> T219626 add DB grants for keystone/nova/nova_api to clouddb2001-dev (codfw1dev) [admin]
2019-04-13 §
18:25 <bd808> Restarted nova-compute service on cloudvirt1015 (T220853) [admin]
2019-04-02 §
19:52 <andrewbogott> installed new base Stretch image. Updated packages, and runs apt-get dist-upgrade on first boot. [admin]
2019-03-29 §
14:34 <andrewbogott> moving tools-static.wmflabs.org to point to tools-static-13 in eqiad1-r [admin]
00:00 <bstorm_> T193264 Added osm.db.svc.eqiad.wmflabs to cloud DNS [admin]
2019-03-25 §
00:40 <bd808> Restarted maintain-dbusers on labstore1004. Process hung up on failed LDAP connection. [admin]
2019-03-21 §
19:32 <andrewbogott> restarting keystone on cloudcontrol1003 [admin]
13:49 <gtirloni> converted openstack cronjobs to systemd timers (T210818) [admin]
00:49 <gtirloni> nfs-exportd interval changes from 60 to 300s (T217086) [admin]
2019-03-15 §
16:00 <gtirloni> increased nscd cache size (T217280) [admin]
2019-03-14 §
19:04 <gtirloni> bstorm started nfsd on labstore1006 (T218341) [admin]
16:42 <gtirloni> published new debian-9.8 image (T218314) [admin]
2019-03-04 §
19:37 <bstorm_> umounted /mnt/nfs/dumps-labstore1006.wikimedia.org across all VPS projects for T217473 [admin]
2019-02-26 §
12:46 <gtirloni> shutdown toolsbeta-sgegrid-master (cronspam) [admin]
2019-02-25 §
10:32 <gtirloni> restarted nfsd on labstore1004 [admin]
2019-02-21 §
09:09 <gtirloni> restarted uwsgi-labspuppetbackend.service on labpuppetmaster1001 [admin]
07:42 <gtirloni> created project cloudstore [admin]
07:36 <gtirloni> deleted wmcs-nfs project [admin]
2019-02-20 §
21:58 <andrewbogott> silencing shinken and disabling puppet on shinken-02 for now [admin]
2019-02-19 §
12:00 <gtirloni> added nagios@icinga2001.wikimedia.org to cloud-admin-feed@ allowed senders [admin]
2019-02-18 §
20:21 <gtirloni> downtimed cloudvirt1020 [admin]
20:12 <gtirloni> ran `labs-ip-alias-dump.py` on cloudservices/labservices servers [admin]
2019-02-15 §
13:10 <arturo> T216239 labvirt1019 has been drained [admin]
12:22 <arturo> T216239 draining labvirt1009 with a command like this: `root@cloudcontrol1004:~# wmcs-cold-migrate --region eqiad --nova-db nova 2c0cf363-c7c3-42ad-94bd-e586f2492321 labvirt1001` [admin]
12:02 <arturo> more nova service cleanups in the database (labvirts that were reallocated to eqiad1) [admin]
11:34 <arturo> T216190 cleanup from nova database `nova service-delete 35` [admin]
03:50 <andrewbogott> updated VPS base images for Jessie and Stretch, now featuring Stretch 9.7 [admin]
2019-02-11 §
18:13 <gtirloni> cleaned old metrics data in labmon1001 T215417 [admin]