I currently have a portion of my backups on S3, with a life-cycle policy that includes moving the objects to Glacier after a period of time. This makes the storage much cheaper ($0.01/GB/Mo from $0.03/GB/Mo – Source), but has the downside that objects require a 4-hour restore period before they can become available for download. I have had need for some objects quickly, and so the 4-hour restore time isn’t worth the savings. Unfortunately, once an object has had this life-cycle applied to it, it can only be temporarily restored. In order to make it a standard object again, you have to download it, delete the Glacier object, and then re-upload it. Unfortunately, doing it all wasn’t quite as straightforward as I thought it might be. But, (I think) I figured out a way to get it done rather painlessly.
I’m going to be using s3cmd and a few cron jobs to automate this.
First, get s3cmd version 1.5. This version supports initiating restores on the Glacier objects. You can recursively initiate a restore on every object in the bucket, but when it hits a non-Glacier object it will stop. You can also use s3cmd to initiate a download of all the objects in the bucket, but when it hits a Glacier object, the download will stop. And you will end up with a zero-byte file. (Hey s3cmd developers, would you mind fixing this behavior, or at least writing in something to force progression on a failure, so we can walk through the entire bucket in one go?)
The solution had to involve initiating restores, waiting at least 4 hours for the restore, then going back for the restored data and deleting it from the buckets, then deleting any zero-byte files, and then doing it all over again later.
Ain’t nobody got time for that. Except cron. Cron has plenty of time for that.
First of all, make sure you have s3cmd installed and configured (with
s3cmd --configure). Then you can configure the following script to run every 4 hours. I’m not going to go into much detail on this. If you’re familiar with s3cmd and Amazon S3/Glacier, you can probably figure out how it works. I wrote it as a short-term fix, but it’s worth sharing.
#!/bin/bash # This script should be fired every 4 hours from a cron job until all # data from the desired bucket is restored. # Requires s3cmd 1.5 or newer # Temp file TEMPFILE=~/.s3cmd.restore.tmp # Bucket to restore data from. Use trailing slash. BUCKET="s3://bucketname/" # Folder to restore data to. Use trailing slash. FOLDER="/destination_folder/" # Because of the way s3cmd handles errors, we have to run in a certain method # 1: download/delete files from bucket, # 2: run restore on the remaining objects # 3: Do housekeeping on the downloaded data if [ ! -f $TEMPFILE ] then touch $TEMPFILE echo === Starting Download Phase s3cmd -r --delete-after-fetch --rexclude "/$" sync $BUCKET $FOLDER echo === Starting Restore Phase s3cmd -r -D 30 restore $BUCKET echo === Starting cleanup # s3cmd doesn't delete empty folders, and can create empty files. Clean this up. find $FOLDER -empty -delete # but it might accidentially delete the target directory if the download didn't # happen, so we have to fix that now mkdir $FOLDER rm $TEMPFILE fi
Note that restore, download, and delete operations can incur extra costs. Be aware of that before proceeding.
So that’s it. I *should* have my entire S3 bucket downloaded completely within the next few days, and then I can migrate to what I hope is a more simplified archiving plan.
My old laptop, a Dell Latitude E6510, has a bad DVD drive. It won’t read most discs, and almost everything I try to burn, fails. I purchased an external USB drive to use with it, and that works well. It was cheaper than replacing the drive itself. Replacing the drive is straightforward, I just don’t want to spend the money right now to get a new drive.
About a week ago I wanted to install some software from a CD (Who does that anymore, right?) on my Asus tablet, which lacks an optical drive, but has a USB port. I plugged the external USB drive into the port, and put the disc in it. Shortly after, the drive kept disappearing from ‘My Computer’, I kept hearing the Windows disconnect and reconnect sounds. It wouldn’t stay ‘connected.’
My son likes to get into things that he shouldn’t, so I nearly chewed him out for it straight away, but after taking a bit to calm down, I decided to troubleshoot. I thought it was a bad cable at first, so I tried a few different cables. No change. Tried wiggling the USB port to see if the tablet port was faulty, no change. I plugged the drive back into the Dell, and it worked without a problem. This got me thinking.
After some of that thinking, I finally figured out what the issue was, and I’m posting this to see if anyone else can figure it out too, because you’ll learn something from my ordeal. Something that may surprise you.
This is a photo of the bottom of the USB drive, showing the label and the USB plug. Most of the information you need to figure this out is in this photo. You may need to go and do a web search if you’re missing a piece of information.
Post your comments, questions, or guesses in the comments below. The first person who figures it out will get recognition, and I’ll clarify the answer if it needs any.
Hint: Again, almost everything you need to know to figure this out is in the photo of the USB drive. Almost. The rest is a quick web search away.
Hint: The Dell laptop has USB 2.0 ports. The tablet has USB 3.0. This is not the reason, although it’s going in the right direction.
Guess/Hint: Someone guessed that the drive was made on April Fool’s Day. A clever guess sir, but not correct. You are looking in the right area.
Guess: Someone guessed that I might have been using the microUSB “On The Go” (OTG) port. A good guess, as this tablet has one of those ports as well, but no. I was using the USB 3 port that’s on the tablet’s keyboard dock. I did just try using the OTG port instead of the USB 3.0 port and got the same result. I am not surprised.
Here are photos. Click any for a larger image:
Answer: My friend Gary got the idea that it might have been a power issue, but cwyenberg went so far as to point that 1.6A is a lot for a USB draw.
Explanation: From Wikipedia:
|USB 1.x and 2.0||500 mA[a]||5 V||2.5 W|
|USB 3.x||900 mA[b]||5 V||4.5 W|
USB devices can only ask for so much power, (up to 500mA for a USB 2.0 device), and 1.6A is far beyond that. This drive can’t be expected to work under all circumstances according to the USB specification. The reason that it worked on the Dell laptop is probably that Dell is allowing the device to draw more power for the sake of compatibility. This is why you see external hard drives with Y-cables — the device can pull from 2 USB ports to get more power and still adhere to the USB standard, allowing the device to be expected to work under all circumstances. Interestingly enough, the drive does work with a powered USB hub rated for 2.5A. So, if you’re having a similar issue as me, try getting a powered USB hub.
iManufacturer : 0x01 Language 0x0409 : "Hitachi-LG Data Storage Inc" iProduct : 0x02 Language 0x0409 : "Portable Super Multi Drive" ... MaxPower : 0xFA (500 mA)
Thanks for reading (and for your responses)!
A reader got in touch with me regarding my previous post, Quick sh script cronjob to fix user homes permissions on Synology. That script was initially intended to fix user homes file ownership, but this reader shared a script that uses the synoacltool to fix the Access Control List on directories.
A few thoughts regarding this script:
First, it was mentioned that these issues may be fixed in the latest DSM release. If you’re still experiencing file ownership and permissions issues, please feel free to use the solution linked to above or posted below.
Second, the script linked to above and the script below take different approaches on the problem. You may find a solution in one, or you may elect to use both.
Third, it was mentioned that this was a “one and done” solution. Due to the changing nature of filesystem content, I don’t believe that to be the case. You may want to save this as a sh script and run it as a scheduled task, or you may want it to run on every boot up. If you decide you want to run it on every boot, edit (or create) the file /etc/rc.local, and paste the below. I can’t say for certain whether this script is preserved on an upgrade, though this page strongly suggests that it would be preserved.
I don’t have a Synology unit right now to test this on, so I can’t offer any insight other than what I’ve shared above.
Here’s the script:
synouser --enum all > user.list
sed -i 's/\\/\\\\/g' user.list
cat user.list | while read line
echo -n "$line: "
USERDIR=`synouser --get "$line" | grep "User Dir"`
if [ $? != 0 ]; then
echo "user: [$line] not found"
HOMEPATH=`echo "$USERDIR" | cut -d'[' -f2 | cut -d']' -f1`
synoacltool -get-archive "$HOMEPATH" | grep is_support_ACL > /dev/null 2>&1
if [ $? != 0 ]; then
echo "[$HOMEPATH] not support ACL or not exist"
synoacltool -get "$HOMEPATH" | grep -F "user:$line:allow:rwxpdDaARWcCo:fd--" > /dev/null 2>&1
if [ $? = 0 ]; then
echo "[$HOMEPATH] exist user's Full Control ACL"
synoacltool -add "$HOMEPATH" "user:$line:allow:rwxpdDaARWcCo:fd--"
Any feedback is welcome and appreciated. Thank you!
I’ve seen issues with the Asus T100 where the Wi-Fi will frequently not reconnect after coming out of standby, hibernate, or a power off. Toggling airplane mode or rebooting the device will fix it, but not always the first time.
I found a fix for this, after reviewing the fix for frequently disconnecting Bluetooth devices, and the issues appear related.
Again, go into device manager and right-click the wireless network adapter, and click Properties. Go to the Advanced tab and change Minimum Power Consumption to Disabled.
After doing this, no more Wi-Fi issues!
I found this fix specifically when looking for a way to fix disconnection issues with a Bluetooth mouse, but it fixes a multitude of issues as well.
Asus T100 frequently disconnects Bluetooth devices in an overly-aggressive effort to save power. Fix this by disabling Bluetooth power management.
First, make sure you have all the up-to-date drivers from Windows Update and from ASUS Live Update.
Next, open Device Manager and right-click Bluetooth Radio, then click Properties, Power Management tab, and uncheck “Allow the computer to turn off this device to save power” as shows in the screenshot below.
That’s all. After doing that, no more BT device disconnects!
A frequently used tool to identify Windows issues is the System File Checker, or sfc. It’s usually run as such:
This will scan through your system files and tell you if any files are corrupt. But — is there an easy way to repair those corrupt files, other than doing a full re-install? Turns out, there is: Connecting to the Internet, and using dism, as shown below:
dism /online /cleanup-image /restorehealth
See below for a copy-paste log that I did on the system I’m currently using to write this post. SFC identified corrupt files, I ran dism, and then re-ran sfc.
C:\WINDOWS\system32>sfc /scannow Beginning system scan. This process will take some time. Beginning verification phase of system scan. Verification 100% complete. Windows Resource Protection found corrupt files but was unable to fix some of them. Details are included in the CBS.Log windir\Logs\CBS\CBS.log. For example C:\Windows\Logs\CBS\CBS.log. Note that logging is currently not supported in offline servicing scenarios. C:\WINDOWS\system32>dism /online /cleanup-image /restorehealth Deployment Image Servicing and Management tool Version: 6.3.9600.17031 Image Version: 6.3.9600.17031 [==========================100.0%==========================] The restore operation completed successfully. The component store corruption was repaired. The operation completed successfully. C:\WINDOWS\system32>sfc /scannow Beginning system scan. This process will take some time. Beginning verification phase of system scan. Verification 100% complete. Windows Resource Protection did not find any integrity violations.
Credits to this forum for having the original post of the fix.
Here’s a question for my Amateur Radio friends out there:
How important are RF power considerations when operating mobile HF?
Here’s why I’m asking: I’m looking into HF, and mobile HF is going to be an option I explore. My owners manual (see below) says to keep the power below 10 watts for CB (29 MHz). I currently operate VHF up to
50 watts with no issues, but I know 2 meter and 10/11 meter are well different.
I’ve heard second- and third-hand stories of people killing their ECMs from high power output, so that’s why I’m throwing the question out. I’m not keen on making my VW into a 5-figure paperweight, and I’m okay with keeping it under 10 watts, but I want to know your thoughts and opinions. Note: I’m looking for information strictly related to the possible effect on the vehicle electronics, not human RF exposure. That’s a different topic entirely.
My intention is to connect the radio’s power supply directly to the vehicle battery (both positive and negative), and use a mount that provides electrical grounding of the antenna as well.
Here’s the page from my owners manual:
So let me know your thoughts and opinions below. Post your call only if you’re comfortable doing so. Please let me know your thoughts, what bands you operate mobile on, what power levels, and what (if any) your installation considerations are.