Running with Ruby

Tag: nas (page 1 of 2)

Synology DMS 6 + Gitlab Docker + Gmail

When you’ll setup your Gitlab instance using Synology DMS 6.0 UI, despite the fact that you’ve provided all the Gmail credentials, you will notice that it does not send any emails. You probably end up with a message similar to this one when:

EOFError: end of file reached

It may sound a bit enigmatic, but in general it means that your Gmail setup is wrong. Unfortunately you cannot change it using the UI. Fixing this requires a SSH connection.

Warning: I’m not responsible for any damages or injury, including but not limited to special or consequential damages, that result from your use of this instruction.

Stop your Gitlab using DMS UI

First, you need to stop your Gitlab instance via DMS UI:

  1. So to Package Manager
  2. Select Gitlab
  3. Select “Actions”
  4. Click on the “Stop” button

gitlab

Change your synology_gitlab.config via SSH

This one is a bit harder. Here are the th ings you need to know before you start:

  • How to log in via SSH into admin account
  • How to install (if you don’t have) vim (or any other console text editor)
  • How to navigate in a Linux shell

So, if you know this things, you are ready to go!

  1. Log in into your Synology server via SSH as admin user
  2. Become a root (sudo su + password)
  3. Go to /usr/syno/etc/packages/Docker (cd /usr/syno/etc/packages/Docker)
  4. Edit synology_gitlab.config (vim synology_gitlab.config)
  5. Replace or add given config options into the env_variables section
  6. Save the file

Those are the options you need to set:

{
  "key":"SMTP_ENABLED",
  "value":"true"
},
{
  "key":"SMTP_DOMAIN",
  "value":"www.gmail.com"
},
{
  "key":"SMTP_HOST",
  "value":"smtp.gmail.com"
},
{
  "key":"SMTP_PORT",
  "value":"587"
},
{
  "key":"SMTP_USER",
  "value":"your user name"
},
{
  "key":"SMTP_PASS",
  "value":"your password"
},
{
  "key":"SMTP_OPENSSL_VERIFY_MODE",
  "value":"none"
},
{
  "key":"SMTP_AUTHENTICATION",
  "value":"login"
},
{
  "key":"SMTP_STARTTLS",
  "value":"true"
}

The whole file should look more or less like that:

{
  "cap_add":[

  ],
  "cap_drop":[

  ],
  "cmd":"",
  "cpu_priority":0,
  "ddsm_bind_share":"",
  "devices":[

  ],
  "enable_publish_all_ports":false,
  "enabled":true,
  "env_variables":[
    {
      "key":"GITLAB_HOST",
      "value":"gitdomain"
    },
    {
      "key":"GITLAB_PORT",
      "value":"port"
    },
    {
      "key":"GITLAB_SSH_PORT",
      "value":"port"
    },
    {
      "key":"GITLAB_EMAIL",
      "value":"email"
    },
    {
      "key":"DB_TYPE",
      "value":"mysql"
    },
    {
      "key":"DB_HOST",
      "value":"172.17.0.1"
    },
    {
      "key":"DB_NAME",
      "value":"gitlab"
    },
    {
      "key":"DB_USER",
      "value":"gitlab"
    },
    {
      "key":"DB_PASS",
      "value":"password"
    },
    {
      "key":"GITLAB_SECRETS_DB_KEY_BASE",
      "value":"secret"
    },
    {
      "key":"SMTP_ENABLED",
      "value":"true"
    },
    {
      "key":"SMTP_DOMAIN",
      "value":"www.gmail.com"
    },
    {
      "key":"SMTP_HOST",
      "value":"smtp.gmail.com"
    },
    {
      "key":"SMTP_PORT",
      "value":"587"
    },
    {
      "key":"SMTP_USER",
      "value":"email"
    },
    {
      "key":"SMTP_PASS",
      "value":"password"
    },
    {
      "key":"SMTP_OPENSSL_VERIFY_MODE",
      "value":"none"
    },
    {
      "key":"SMTP_AUTHENTICATION",
      "value":"login"
    },
    {
      "key":"SMTP_STARTTLS",
      "value":"true"
    }
  ],
  "exporting":false,
  "id":"id",
  "image":"sameersbn/gitlab:8.6.2",
  "is_ddsm":false,
  "is_package":true,
  "links":[
    {
      "alias":"redisio",
      "link_container":"synology_gitlab_redis"
    }
  ],
  "memory_limit":0,
  "name":"synology_gitlab",
  "pin":false,
  "port_bindings":[
    {
      "container_port":80,
      "host_port": port,
      "type":"tcp"
    },
    {
      "container_port":22,
      "host_port": port,
      "type":"tcp"
    }
  ],
  "privileged":false,
  "shortcut":{
    "enable_shortcut":false,
    "enable_status_page":false,
    "enable_web_page":false,
    "web_page_url":""
  },
  "volume_bindings":[
    {
      "host_volume_file":"/docker/gitlab",
      "mount_point":"/home/git/data",
      "type":"rw"
    }
  ]
}

Start your Gitlab using DMS UI

Procedure is exactly the same as stopping except the last step: just click on “Run” button instead of “Stop”.

Retry all failed tasks in the Gitlab Sidekiq panel

  1. Log in to your Gitlab admin account
  2. Go to the “Admin area” (right top corner icon) – /admin
  3. Click on “Background jobs” (left bottom corner) – /admin/background_jobs
  4. Go to “Retries”
  5. If you have any, click on “Retry all” to reexecute all the failed tasks

sidekiq

And that’s all! If you didn’t make any mistakes, your Gitlab emails should be sent via provided Gmail account.

Recovering QNAP NAS lost data when NAS not starting properly

QNAP is a company that designs great network attached storages (NAS). Unfortunately, even their NAS can crash. Mine did. Before you get to how to recover the lost data, here’s my NAS and RAID spec (so that you can understand what and why I did):

  • QNAP TS-410U
  • RAID5
  • 4 HDD (/dev/sda, /dev/sdb, /dev/sdc, /dev/sdd)
  • Approximately 1.4 TB of data
  • Fortunately I had the most important data already backuped somewhere else (less pressure and stresses during fixing)

And this is what happened to it:

  1. NAS software update (for 1 week everything was working fine)
  2. NAS rejected one of my HDDs (/dev/sda) due to SMART status.
  3. RAID5 is now in degradation mode.
  4. Broken HDD has been removed (not replaced!).
  5. NAS has been shutdown (I didn’t plan to use it so I turn it off for 2 weeks – just in case).
  6. NAS would not boot with HDDs inside (well it would boot but it didn’t get an IP address, so that I could get to it).
  7. NAS is not reachable at all (despite the fact that it seemed to work just fine).
  8. Basic system reset (3s) didn’t help at all (still no network connection).

Booting without any hard drives

You won’t be able to do anything, unless you manage to get online with your QNAP. If it’s just a software issue (which was in my case), follow these instructions:

  1. Force shutdown of your NAS (press power button for 10 seconds)
  2. Remove all the hard drives
  3. Turn on your NAS by pressing power button
  4. Once it is ready (it beeps), perform a basic system reset
  5. Restart your NAS (either by performing shutdown or by disconnecting power)
  6. Boot it again
  7. You should be able to reach the following website: http://your-nas-ip-address:8080/
  8. Unfortunately you don’t have any hard drives connected, so no data recovery yet ;)

No hard drives and no setup equals no way to recover data

Udls9W0

Before you attach our hard drives and restore RAID, you need to know one thing: QNAP that is not a setup with at least 1 HDD, won’t provide you with any tools like scp or rsync. You will be able to examine your HDDs (there’s mdadm luckily), but you won’t transfer your data via LAN. All network tools are only available once you perform a full setup. Also keep in mind, that you should perform a whole new installation with your RAID hard drives unplugged (just in case).

Spare HDD to the rescue

Make your NAS available via SSH with all the tools you need.
To do this, you will have to have one spare hard drive (any SATA HDD will be ok). Now:

  1. Turn off your NAS.
  2. Plug in your HDD.
  3. Make sure your RAID HDDs are unplugged.
  4. Power on your NAS.
  5. Once it boots, go to admin page and perform a quick setup.
  6. Now you should be able to connect to it via SSH (ssh admin@your-nas-ip) user: admin, password: admin
  7. Once you connect, check if you have the following commands available: rsync, scp, mdadm

Reassembling RAID5 and mounting it to recover data

I used the first HDD slot for a temporary “rescue” HDD (/dev/sda). So it won’t be included when I will reassemble the rest of HDDs.

Before you assemble anything, you need to check if there’s valid RAID data on each of the remaining HDDs:

# You will have also 
mdadm --examine /dev/sdb3
mdadm --examine /dev/sdc3
mdadm --examine /dev/sdd3

For each of them, you should see something like that:

/dev/sdc3:
          Magic : a92b4efc
        Version : 00.90.00
           UUID : 0fcde09f:5258ded4:4c22c8ef:89a53221
  Creation Time : Sat Mar  9 21:13:27 2013
     Raid Level : raid5
  Used Dev Size : 1951945600 (1861.52 GiB 1998.79 GB)
     Array Size : 5855836800 (5584.56 GiB 5996.38 GB)
   Raid Devices : 4
  Total Devices : 3
Preferred Minor : 1

    Update Time : Sun Feb  1 13:32:54 2015
          State : active
 Active Devices : 3
Working Devices : 3
 Failed Devices : 0
  Spare Devices : 0
       Checksum : fb959cff - correct
         Events : 0.1608150

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     2       8       35        2      active sync   /dev/sdc3

   0     0       0        0        0      removed
   1     1       8       19        1      active sync   /dev/sdb3
   2     2       8       35        2      active sync   /dev/sdc3
   3     3       8       51        3      active sync   /dev/sdd3

Now reassembling:

# Use /dev/mdNR that is not taken already
mdadm --assemble /dev/md1 /dev/sdb3 /dev/sdc3 /dev/sdd3

After executing this command, your RAID should be assembled and ready to mount:

mkdir /share/QSAVE
mount -t ext4 /dev/md1 /share/QSAVE

If everything went ok, you can see your data, when you’ll:

cd /share/QSAVE
ls

Local backup

If you used a decent “rescue” HDD, you can now use it as a backup hard drive for all of your NAS data (as long as it is big enough):

mkdir /share/HDA_DATA/backup
rsync -rv /share/QSAVE/ /share/HDA_DATA/backup/

Remote backup

You can also backup your NAS remotely:

mkdir ./qnap_backup
rsync -rv --exclude=".*" admin@your-nas-ip:/share/QSAVE/ ./qnap_backup

Also keep in mind, that even when you have RAID1, RAID5, RAID10 and so on, it is still worth having an external backup of all of your data.

Olderposts

Copyright © 2018 Running with Ruby

Theme by Anders NorenUp ↑