Time drift in Arch Linux

Have you ever looked at the clock on your computer, then looked at the clock on your mobile device and wondered why they differed?  The easy answer is time drift.  The realtime clock (RTC) in your computer, as well as mobile devices like smartphones, tablets, laptops, watches, etc operates on its own timeframe.  It does not conform to any universal standard.  This is why utilities such as NTP and timedatectl were created.

The timedatectl utility is specifically designed for Arch Linux as an easy-to-use utility for the Network Time Protocol (NTP), but NTP is generally controlled a bit more directly in Ubuntu, Redhat and other distributions.  I'm going to focus on timedatectl for this particular entry.

First, open up a terminal.

Next, let's get an idea of what's currently going on with the time and date situation.  Just run the following command:

timedatectl status

This tells you the various times, which timezone you're set to, whether the network time is on, if NTP is turned on and whether or not the RTC is set to the local TimeZone.  Since we want to correct for time drift, let's get NTP synchronized.  In order to do that, we need to turn Network time on.  Let's start by checking out our options.

There is a set-ntp option.  It states that it needs a BOOL, or boolean, which can be one of two different entries.  In this case, 'yes' and 'no' are a pair of very human conditions that we can work with.  It is important to perform this function as a standard user.  It will ask you to authenticate.  This is for security purposes.

Once that is completed, you can check to see the current status.

Now, the Network time is set to 'on', and all three clocks are set to the current time.  This should reflect from other devices as well.

By maintaining the current time on your desktop or portable computer, you can avoid delays.  Happy time!

Compilation of OpenCV programs

Compiling programs written in C++ is usually a fairly easy and non-complex task to complete.

% g++ -o program program.cpp

However, OpenCV (Open Source Computer Vision) employs a much more complex set of libraries that are necessary to reference object compilation.  While a good amount of documentation exists to manipulate and generate a useful program to generate desired outcomes, some of it remains an exercise to the programmer to know how to get it setup and running.  Just about anyone that has that level of knowledge can their code to work just fine.  However, sometimes the obvious slips by.

So, you're working on your program and it's based off of some known, good working code.  You set your parameters and hit the compilation string.

/tmp/ccSv9hPY.o: In function `MatchingMethod(int, void*)':

/home/user/Project/strimaker/template/MatchTemplate_Demo.cpp:87: undefined reference to `cv::imshow(cv::String const&, cv::_InputArray const&)'

/home/user/Project/strimaker/template/MatchTemplate_Demo.cpp:88: undefined reference to `cv::imshow(cv::String const&, cv::_InputArray const&)'

/tmp/ccSv9hPY.o: In function `main':

/home/user/Project/strimaker/template/MatchTemplate_Demo.cpp:37: undefined reference to `cv::namedWindow(cv::String const&, int)'

/home/user/Project/strimaker/template/MatchTemplate_Demo.cpp:38: undefined reference to `cv::namedWindow(cv::String const&, int)'

/home/user/Project/strimaker/template/MatchTemplate_Demo.cpp:42: undefined reference to `cv::createTrackbar(cv::String const&, cv::String const&, int*, int, void (*)(int, void*), void*)'

/home/user/Project/strimaker/template/MatchTemplate_Demo.cpp:46: undefined reference to `cv::waitKey(int)'

collect2: error: ld returned 1 exit status

Now you've got a slew of problems and a non-existent binary.

Linking the libraries is fairly simple, as long as you know what to link against.

 g++ -o MatchTemplate_Demo MatchTemplate_Demo.cpp -g -O2 -lopencv_imgcodecs -lopencv_imgproc -lopencv_core -lopencv_highgui

Here, I'm linking against anything the headers require.  One way to figure out what libraries to use is through the use of the Autotools program.  Unfortunately, OpenCV relies on CMake, and so one must be able to distinguish between them in order to avoid wasting time on a problem that may not need to exist.

Getting MRTG to work in ArchLinux

It's been a long time since I ran MRTG, the Multi Router Traffic Grapher.  So, I figured it was time to run it again.  Using Arch Linux, it's pretty simple to get it installed and configured.  However, the current model does not come with a mrtg.service file.

In fact, no one has written one!  It's supposed to be ran by hand.  This kind of dulls things, since a server ought to be able to be brought back online and it should "just work" with little human intervention.  So, I'm a bit of a perfectionist when it comes to making things work just right, out of the box.

Creating service files for systemd is not very difficult.  I just used another .service file as a template to work from.

The only other type of service file that I could find was a CentOS oriented control file, and by the looks of it, someone didn't realize that you could run MRTG in daemon mode and not have to worry about rerunning the process every so often.  Still, it was helpful to remember that daemonizing isn't always the necessary way.

[Unit]



Description=Multi Router Traffic Grapher



After=network.target




[Service]



PIDFile=/run/mrtg/mrtg.pid



User=mrtg



Group=mrtg



ExecStart=/usr/lib/systemd/scripts/mrtg.sh



ExecReload=/usr/bin/kill -USR2 $MAINPID



KillSignal=SIGQUIT



KillMode=mixed






[Install]



WantedBy=multi-user.target

You'll notice that I generated a shell script to handle actually starting the daemon.  I was getting errors while trying to load an environment variable into it, so I set it to run its own process:

#!/bin/sh
LANG=C /usr/bin/mrtg --pid-file=/run/mrtg/mrtg.pid --user=mrtg --group=mrtg --daemon /srv/http/mrtg/html/mrtg.cfg

Create /run/mrtg:

# mkdir /run/mrtg

Set the appropriate permissions:

# chown mrtg.mrtg /run/mrtg

When all is said and done, MRTG will run in the background, unattended, and produce beautiful graphs in Arch Linux.

Internet traffic real-time detection using Arch Linux

So the last post got off to a bit of a rocky start.  After doing a bit more research, I discovered that the Snort IDS system was very finite.  There needed to be better integration.  That is where Suricata comes in.  Suricata is meant as a replacement for Snort in certain situations.

Unfortunately, Suricata doesn't appreciate all of Snort's rules.  This can quickly go from frustrating to something else.  Thankfully, many basic commands allow to ease the situation down to a few simple commands.

Let's take a look at what's actually going on in the logs:

[10392] 27/6/2015 -- 16:12:37 - (detect.c:357) <Error> (DetectLoadSigFile) -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET any -> $HTTP_SERVERS $HTTP_PORTS (msg:"ET WEB_SPECIFIC_APPS The Address Book SQL Injection Attempt -- user.php lastname ASCII"; flow:established,to_server; uricontent:"/user.php?"; nocase; uricontent:"lastname="; nocase; uricontent:"SELECT"; nocase; pcre:"/ASCII\(.+SELECT/Ui"; reference:cve,CVE-2006-4575; reference:url,www.securityfocus.com/bid/21870; reference:url,doc.emergingthreats.net/2006007; classtype:web-application-attack; sid:2006007; rev:5;)" from file /etc/suricata/rules/ET-emerging-web_specific_apps.rules at line 2273       [10392] 27/6/2015 -- 16:12:37 - (detect-reference.c:159) <Error> (DetectReferenceParse) -- [ERRCODE: SC_ERR_REFERENCE_UNKNOWN(150)] - unknown reference key "cve". Supported keys are defined in reference.config file.  Please have a look at the conf param "reference-config-file"                                                                                                                                                                                                     [10392] 27/6/2015 -- 16:12:37 - (detect.c:357) <Error> (DetectLoadSigFile) -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET any -> $HTTP_SERVERS $HTTP_PORTS (msg:"ET WEB_SPECIFIC_APPS The Address Book SQL Injection Attempt -- user.php lastname UPDATE"; flow:established,to_server; uricontent:"/user.php?"; nocase; uricontent:"lastname="; nocase; uricontent:"UPDATE"; nocase; pcre:"/UPDATE.+SET/Ui"; reference:cve,CVE-2006-4575; reference:url,www.securityfocus.com/bid/21870; reference:url,doc.emergingthreats.net/2006008; classtype:web-application-attack; sid:2006008; rev:5;)" from file /etc/suricata/rules/ET-emerging-web_specific_apps.rules at line 2274          [10392] 27/6/2015 -- 16:12:37 - (detect-reference.c:159) <Error> (DetectReferenceParse) -- [ERRCODE: SC_ERR_REFERENCE_UNKNOWN(150)] - unknown reference key "cve". Supported keys are defined in reference.config file.  Please have a look at the conf param "reference-config-file"                                                                                                                                                                                                     [10392] 27/6/2015 -- 16:12:37 - (detect.c:357) <Error> (DetectLoadSigFile) -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET any -> $HTTP_SERVERS $HTTP_PORTS (msg:"ET WEB_SPECIFIC_APPS The Address Book SQL Injection Attempt -- user.php firstname SELECT"; flow:established,to_server; uricontent:"/user.php?"; nocase; uricontent:"firstname="; nocase; uricontent:"SELECT"; nocase; pcre:"/SELECT.+FROM/Ui"; reference:cve,CVE-2006-4575; reference:url,www.securityfocus.com/bid/21870; reference:url,doc.emergingthreats.net/2006009; classtype:web-application-attack; sid:2006009; rev:5;)" from file /etc/suricata/rules/ET-emerging-web_specific_apps.rules at line 2275       

Here, I've pulled 3 rejections out of a list of over 10,000.


Yes, over 10,000.

Here is where some basic commands and simple math come into play in order to take a problem the size of Mount Everest and bring it down to the size of a grain of sand.  You can easily notice that each rule being rejected has its own SID.  An SID is an identification token used to identify a specific rule.  Each rule must have an SID for it to be considered by the program.  The problem comes that many of such rules from Snort are not properly formatted to handle Suricata's system.  I am not sure why this is the case, so while it would be proper to fix the rules, it is simpler for the time being to just move the offending rules out of the way.  We're not actually moving them out of the way, but they are being ignored so that the rest of the program may function as it is supposed to.  Once the program is able to run, then fixing the broken rules can begin and things can still run smoothly.  However, there has to be a clear understanding that this is what is going to happen, as many rules are absolutely critical to things running in the desired direction.  Therefore, getting those other rules fixed ASAP will become the next priority, rather than just leaving them to rot in an inefficient and disrespectful manner.

From the Snort manual:

This example is a rule with the Snort Rule ID of 1000983.

    alert tcp any any -> any 80 (content:"BOB"; sid:1000983; rev:1;)

 So here we can see that there is some adherence to the basic format.  Again, getting down to the root cause will be left for another time.  First of all, let's find out just how may offending rules we have:

[root@mebion rules]# grep sid /var/log/suricata.log | sed -e 's/.*sid//g' | sed -e 's/rev.*//g' | sed -e 's/://g' | sed -e 's/;//g' | wc -l                                                                                                  11511                                                                                                                                                                                                                                       

Well now, 11,511 bad rules.  It probably could have been performed a bit more elegantly, but drawing it out to this degree appears to work just perfectly.  Next, we need to know how many rules there are altogether, so that way we can be sure that we aren't throwing ourselves under the bus.  After all, what if there are only 11,511 rules total?

 [root@mebion rules]# grep sid *.rules | sed -e 's/.*sid//g' | sed -e 's/rev.*//g' | sed -e 's/://g' | sed -e 's/;//g'  | wc -l                                                                                                               46520                                                                                                                                                                                                                                        

Okay, 46,520 rules total as of this time.  Now, math:

[root@mebion rules]# bc

 bc 1.06.95 Copyright 1991-1994, 1997, 1998, 2000, 2004, 2006 Free Software Foundation, Inc.
 This is free software with ABSOLUTELY NO WARRANTY.
 For details type `warranty'.
 46520-11511
 35009                                                                                                                                                                                                                  
Good, we've got 35,009 good rules to work with.  11,511 rules doesn't seem so significant now, does it?

Now, according to the documentation, it is possible to ignore those rules:

If you disable a rule in your rule file by putting a # in front of it, it will be enabled again the next time you run Oinkmaster. You can disable it through Oinkmaster instead, by entering the following:

cd /etc/suricata/rules

and find the sid of the rule(s) you want to disable.

Subsequently enter:

sudo nano /etc/oinkmaster.conf

and go all the way to the end of the file.
Type there:

disablesid 2010495

Instead of 2010495, type the sid of the rule you would like to disable. It is also possible to disable multiple rules, by entering their sids separated by a comma.

 Here is where it gets interesting.  We can put the list of all the SID's into a file:

grep sid /var/log/suricata.log | sed -e 's/.*sid//g' | sed -e 's/rev.*//g' | sed -e 's/://g' | sed -e 's/;//g' > /etc/suricata/bad-sids.txt

 Which, of course, gives us each SID on each line.  That's not separated by a comma.  Therefore, we need to reformat.

cat bad-sids.txt | tr '\n' , > fixed-sids.txt

 Okay, now that they have been properly formatted, they can simply be copy & pasted into the configuration file all on one line.  This solves that problem, enjoy!

Firewall with the Raspberry Pi

Figuring out what kind of firewall works best for your situation is never a sweatless task.  For a[n] [un]lucky few, it's a no-brainer.  However, in my case I decided that a firewall needs to do more than just block a few ports.  Nowadays, with the advent of malware and hackers getting into systems via web browsers, it's easier than ever for identity thieves to strike.  So, we need something that will actually work by not only blocking, but alerting us to violations that slip past through ports 80 and 443.  We need a way to identify web traffic, as it can pass through any port.  We also need to determine if the information passing through it is of a certain nature.

If little Johnny is researching his school project, it wouldn't be very productive if he got distracted by visiting other websites.  Indeed, a parent walking in on a situation should have some foresight to what's really going on.  These days, teenagers have more than enough stress to worry about.  The less stressed the parents are, the easier it is on the kids.  But, we also want to make sure that they know who is in charge.  Kids should be able to feel a form of freedom to explore and express themselves.  It is easy for that to become a  problem.  Even if the problem has already occurred, it's not too late to get a remedy in place.  Being pro-active is one of the hardest things a parent can do, since the situation can go sideways in a split second.

So, in my previous blog post, I introduced the concept of using a Raspberry Pi computer to act as a firewall.  Many iterations of this have been successfully implemented.  One of my favorites is called the PiWall.

EDIT: It turned out the instructable was just a copy & paste ripoff of the original blog, and so won't be linked to any further.

In this case, I've made some modifications to the base system.  Since I use flash drives in lieu of hard drives in order to save energy and produce less heat, I've replaced all instances of the Extended Filesystem with Samsung's Flash Friendly Filesystem, or F2FS.  Furthermore, some easy filesystem performance tests prove beyond a shadow of a doubt of Samsung's commitment to the Open Source Community.  In practice, some sequences that once produced lag in EXT4 now provide results instantaneously.  Your Mileage May Vary (YMMV), but overall I am happy with the results.

Until the next update...

Roll the updates

In the coming months, stay tuned for a series regarding traffic monitoring and some detail to creating a system that generates near real-time alerts.

The start of this project began with a simple Linux installation on a Raspberry Pi.  Instead of listing everything out here, I will simply link on to a post that better describes the process of that setup here.

Once that step is finished, the next step will be ready.  Stay tuned this summer for more!

Google Now woes

Do you own an Android powered tablet?  If it's running the latest KitKat, perhaps you've also come across Google's new "Google Now" app that's being advertised.  In theory, I think it has promise.  However, at this time it doesn't really do much to enhance my usage other than to increase my frustration level. 

As a result, I uninstalled it.

This left my icons in a weird state: Most were still there, but many had been replaced with a generic green Android icon.  Some labels had also been stripped away so only the process name that the underlying Linux system knows them as was showing (Such as com.zygote or com.motorola.something!).  How to bring them all back without going through each one?!  Shut the tablet off.  Yup, then turn it back on.  After I did so, my tablet was back to its original setup that I had made for it.

I'm sure someone will come across the same situation, so there you have it!

Getting Netflix working on the LG BP300

After many years, I finally broke down and purchased a Blu-ray player.  I didn't feel the need to spend a lot of money on something just because it was new.  I wanted something that everyone could enjoy and had a bit of extra functionality.  Having a decent DVD collection, it needed to play DVDs as well.  The fact that it has WPA2 encrypted WiFi is a nice bonus.  It also supports Netflix and Vudu, which is akin to having a virtual redbox.

However, getting Netflix to work is not as intuitive to most people as it should be.  I blame this oversight error on LG themselves, since they are responsible for programming the firmware.  The temporary fix is pretty easy to figure out if you're used to bashing away when a problem occurs.  Unfortunately, this leaves those out in the cold feeling like they've wasted their money on a device made by a rather reputable company that see a prompt on the screen with no automatic keyboard input. 

LG support forum.

You see, if you try to sign in to a Netflix account, it will ask for your usual credentials, but it does not provide a way to input them.  My answer to that?  "Enter.". Yes, that big button on the remote control: press it.  If you expect an error, then really, make it prove that there is an error.  In all seriousness, all this really does is bring up a keyboard on the screen that you can use to sign in with.  Once past this hurdle, we were streaming our favorite shows in full HD.

It would be nice if LG would have fixed this problem before getting the units out the door.  It's a great unit, but the technically un-inclined are filling up the support forums and LG's responses are showing that the problem is not being clearly communicated nor understood.

How to install Linux on an IBM Thinkpad 750

Now, before your eyebrow finishes its arch, I wrote this howto over 10 years ago!  However, due to the nature of change over the years, the original URL fell off of the internet and this howto is no longer linked to http://www.linux-on-laptops.com.  Therefore, without further adieu, my HOWTO on How to install Linux on an IBM Thinkpad 750:

Running Linux on an IBM Thinkpad 750 (non-colour/monochrome-like display)

by Scott Prader.
Original Version 0.0.4, November 6, 2001

Previous Version 0.1.0, January 1, 2013

Updated Version 0.2.0, February 2, 2014

---

Well, I'll tell you what I tried, what failed, and what succeeded.

BTW, if you are looking for help on running Linux on other laptops, go this site to the Official "Linux on Laptops" homepage.  For more information on IBM Thinkpad laptops in general, this link has some nice information.

---

The Hardware

What I basically have is what was handed to me by someone who did not want this any longer (he doesn't know much about computers and figures that if it doesn't have color then it's as good as junk.) was an IBM Thinkpad 750, complete with power supply. It has a 486 dx 25 (or 33mhz, Linux still hasn't decided what it wants to be yet), 8 megs of ram, a 170 meg ide hard drive, a black & white display, 2 PCMCIA slots, and the standard 15-pin video (for an external monitor), serial and parallel ports. It also has some sort of sound card but I haven't managed to get it to work yet.. from what I heard, it would require a boot to dos or win95 to properly initialize it and then a loadlin into linux.. not exactly a route i wish to take.  I was also fortunate to have another friend with a 3com 3c589 Etherlink III PCMCIA Network Card to use.

What I tried

Redhat did not give the results that I was looking for (it wouldn't boot a full kernel), so I tried Slackware and it was problematic as well.

What failed

Slackware did not work right away.  It would boot the boot floppy just fine and ask for the root floppy (which is normal behavior) but once it started to run it's rc scripts, it found it could not fork any longer.

What worked

A bunch of people on the internet claimed that Debian is the best Linux distro out there.. I was skeptical, but adventurous.. I didn't know what else to do but to give it a go.. so I fished thru hundreds of floppy disks to find 7 working (for the most part) floppies to be used. See, I didn't have a Debian CD at the time but I did have a cable modem, so downloading distro's of any OS off the internet is not much of a problem.  However, Debian took a lot of the guess work out of that.. there is a 7-disk method with which you can install debian from virtually any way: floppy, cdrom, ftp, NFS... so I went with the floppy method.. i knew about the apt-get program that came with it so i wasn't too upset with a minimal installation.  So I booted up the first 2 floppies.  To my surprise, the standard debian floppy disk installation method comes fully complete with PCMCIA support.  It managed to detect the 3c589 just fine.  After installing all of the floppies, I rebooted and was pleased to find a 2.0.36 kernel booting happily.

In Conclusion...

Getting the base OS of Debian installed was a snap.  I then gave it an IP address using ifconfig+route and used Debian's neat config scripts to configure the network.  I plugged some cat5 into the 3c589, plugged it into the hub, told my server about it, and soon enough it was talking to the net!  I used the apt-get program to install what i needed off of the internet.  I didn't need a cdrom for this; I could update it all off the internet with ease.  Considering the hardware limitations, it sure beat hours and hours of compiling.  I tried to get X working but it doesn't seem to like the b&w screen.  However, on an external monitor it will work.  After I installed several things like a compiler, perl and a few other useful things used to program, I still had more than 50 megabytes of hard drive space free. I've taken it with me a lot and when i don't, it sits on top of the TV upstairs for me to use while watching TV.

New Information [ August, 2000 ]

Well, considering it's been nearly a year now without any updates to this page, I figured it was about time.  So, here are some quick notes that i have compiled to make life a little easier for people, along with some tips on newer kernels and distributions:

When booting from a Debian installation (or any linux floppy disk) be sure you specify: floppy=thinkpad otherwise you will run into nasty problems. 

Debian Potato 2.2 has _FINALLY_ been released!! So what's the dilly-o? Welp, if you've ever tried to boot a standard set of boot/root floppies from potato onto a 750, you KNOW it won't work, even with floppy=thinkpad. why? VFS kernel panics when u insert the root floppy. Sooo, the solution is to snag the compact kernel image and drivers and write these over to your floppies... simply create your normal boot floppy, mount it, replace the 'linux' kernel file, unmount it, then go boot it. Suddenly, you can install Potato on your Thinkpad 750. Of course, the drivers must be installed onto the appropriate floppy as well. 

I somehow found myself back in Silicon Valley last week at the LWE and an old BBS friend had some thinkpad stuff laying around.. so now my 750 is running 20 megs of ram (as opposed to 8) and has an 810 meg hard drive (as opposed to 170 megs), so it is much speedier and thus more useful. :) 

Pictures!! Got one up for now, I'll get some more up later. 

TODO

At this point, I still have yet to get X to behave properly, although I have gleamed some information from the video card when typing 'X' on its own and reading the server messages. I also recently stumbled across one of IBM's sites that has complete PDF and, for some, Bookmanager documentation for legacy IBM Thinkpads at http://web.archive.org/web/20010208214159/http://www3.pc.ibm.com/cgi-bin/bookmgr/BOOKS/tpvol2/CCONTENTS.  The documentation appears to include the 750, but after messing around in ftp.pc.ibm.com/pub/pccbbs/mobiles, I came across 750_tips.txt which should help shed some light on anything 750-related. Also, a PDF file (viewable just fine with xpdf) I found contains some information on the 750 (as well as the 720c, one of which I was handed last week from a friend I know from online.. I'm SUCH a hardware geek *g*).

UPDATE!!! [ November, 2001 ]

So I'm sitting here messing around with traffic shaping when I receive an email from Jeremy Ferris thanking me for putting this site up. A week passes and he responds with an XF86Config file for the 750. I have yet to try it out but I certainly will soon; here it is:

# Refer to the XF86Config(4/5) man page for a description of the format
Section "Files"
    FontPath "/usr/X11R6/lib/X11/fonts/misc/"
    FontPath "/usr/X11R6/lib/X11/fonts/Speedo/"
    FontPath "/usr/X11R6/lib/X11/fonts/75dpi/"
    FontPath "/usr/X11R6/lib/X11/fonts/100dpi/"
    RGBPath "/usr/X11R6/lib/X11/rgb"
EndSection
Section "ServerFlags"
EndSection
Section "Keyboard"
    Protocol "Standard"
    AutoRepeat 500 5
    ServerNumLock
    LeftAlt     Meta
EndSection
Section "Pointer"
    Protocol "ps/2"
    Device "/dev/mouse"
    Emulate3Buttons
EndSection
Section "Monitor"
    Identifier "TP Monitor"
    VendorName "IBM"
    ModelName "TP750"
    BandWidth 80        # this may not be correct
    HorizSync 35.38
    VertRefresh 67.38
    ModeLine "640x480" 28.3 640 672 768 800 480 490 492 525
EndSection
Section "Device"
    Identifier "IBM Device"
    VendorName "Unknown"
    BoardName "Unknown"
    #Chipset "wd90c30"
EndSection
Section "Screen"
    Driver "vga16"
    Device "IBM Device"
    Monitor "TP Monitor"
    Subsection "Display"
        Modes "640x480"
        ViewPort 0 0
        Virtual 800 600
    EndSubsection
EndSection

now, this file worked under slackware but I had to change the Device under 
"Pointer" to: /dev/psaux to get it to run for Debian.

My initial problems with getting the Debian setup going all revolved around 
faulty disks! grr! I think that I did not encounter this problem with 
slackware because I used dd differently. I think that maybe the Debian way 
(that is the way one is told to use dd in the installation guide) for using 
dd doesn't check the disks for errors... I could be wrong and I have not 
tested the theory. 

I have since compiled a new kernel on the 750 and am trying to set it up as a 
firewall/proxy.

once again, thanks for having your how-to up, it was helpful.

Jeremy

RTLSDR and High Altitude Ballooning

As I mentioned earlier, I have a vested interest in Amateur Radio and High Altitude Ballooning.  This past summer, I came across the existence of a new device called the RTLSDR, or RealTek Software Defined Radio.  It's a little USB device that you can plug a pigtail to an antenna into.  Mine came with a little magnet-mount antenna, but it's not very big, therefore it gets poor reception.  To remedy this, I acquired an MCX-to-SMA adapter, which I was able to link to an SMA-to-SO239 adapter, and finally onto a groundplane antenna that I built earlier this year and installed outside.  Reception increased significantly, as a result.  While it is not very efficient at picking up satellite transmissions, it's terrific for picking up AM/FM radio and Ham Radio transmissions.  Since APRS generally works in the area of 146Mhz, and that is what the groundplane antenna was built for, I was in luck.

A Groundplane antenna.  This is not mine, it belongs to KC8YVF.  This looks similar to mine.

Here is the completed groundplane antenna as I am holding it, before it was installed.

Recently, the California Near Space Project sent a high altitude balloon across the Atlantic Ocean.  It came to rest somewhere in the western portion of the Mediterranean Sea, breaking a world record.  Two days ago, they sent another long duration flight up.  Now that I knew about it, I was able to prepare.  I set GQRX to listen on the APRS frequency of 144.390Mhz.  Since we get some APRS traffic around, I was able to test it out to see how big of a window to set it to.  15khz seemed about right for the packets I was receiving, and since I wasn't getting any mangled packets, I figured it was a good place to keep it at.  Usually I run GQRX for a short amount of time - usually an hour or less.  I typically don't keep it running, as it is prone to the occasional crash.  However, since I was keeping it in one place, I didn't think that a crash was likely, so when CNSP-18 was over Colorado around 6:00AM CDT, I turned down my master volume after setting everything up, turned off the monitor and left for the day.  When I got home in the afternoon, I had a lot of traffic logged.  I didn't see K6RPT-12 right away, so I saved a copy of the log and searched it.  Sure enough, there were more than a few handfuls of telemetry!

Ubuntu 12.04 with Google Chrome running in the background and GQRX running in the foreground, complete with packet decoder window.  Enlightenment 17 is the window manager of choice.

Below is the traffic that I was able to collect.  It looks like there may have been some congestion on the APRS network since received packets were spaced apart about 10 minutes or so, while tracking via aprs.fi were generally about 2 minutes apart.  However, as the balloon got closer, the received signal grew, thus allowing less packet loss.  Conversely, as the balloon floated away toward Detroit, the number of packets decreased.  A lot of factors go into good and bad signals.  Distance is usually the key factor, but things such as wind gusts can throw some monkey wrenches into the works.


07:49:58$ fm N0AN-0 to APU25N-0 via W0ISU-0,WIDE2-1 UI^ PID=F0
          }KC8SFQ-14>APWW10,TCPIP,N0AN*::K6RPT-12 :?APRSP
07:50:29$ fm N0AN-0 to APU25N-0 via W0ISU-0,WIDE2-0 UI^ PID=F0
          }KC8SFQ-14>APRS,TCPIP,N0AN*::K6RPT-12 :?APRSP
07:50:42$ fm K6RPT-12 to APBL10-0 via W0ISU-0,WIDE2-0 UI^ PID=F0
          !4000.26N/09736.01WO082/150/A=103726V1D4    CNSP-18    CNSP-Inc.com.700
08:01:53$ fm K0SXY-0 to APU25N-0 via WA0ROI-1,W0ISU-0,WIDE2-0 UIv PID=F0
          }KC6VVT>APWW10,TCPIP,K0SXY*::K6RPT-12 :?APRSP
08:01:54$ fm N0AN-0 to APU25N-0 via W0ISU-0,WIDE2-1 UI^ PID=F0
          }KC6VVT>APWW10,TCPIP,N0AN*::K6RPT-12 :?APRSP
08:01:56$ fm N0AN-0 to APU25N-0 via W0ISU-0,WIDE2-1 UI^ PID=F0
          }KC6VVT>APRS,TCPIP,N0AN*::K6RPT-12 :?APRSP
08:10:44$ fm K6RPT-12 to APBL10-0 via W0ISU-0,WIDE2-0 UI^ PID=F0
          !4007.27N/09633.39WO072/158/A=107579V1E5    CNSP-18    CNSP-Inc.com.700
08:27:06$ fm K6RPT-12 to APBL10-0 via W0ISU-0,WIDE2-0 UI^ PID=F0
          !4016.34N/09553.40WO066/204/A=000095V1F6    CNSP-18    CNSP-Inc.com.700
08:50:42$ fm K6RPT-12 to APBL10-0 via W0ISU-0,WIDE2-0 UI^ PID=F0
          !4024.13N/09412.49WO050/142/A=000101V200    CNSP-18    CNSP-Inc.com.700
08:52:34$ fm K6RPT-12 to APBL10-0 via WIDE2-1 UI^ PID=F0
          !4047.54N/09409.75WO069/174/A=110837V211    CNSP-18    CNSP-Inc.com.700
08:54:34$ fm K6RPT-12 to APBL10-0 via WIDE2-1 UI^ PID=F0
          !4031.37N/09405.15WO068/159/A=000101V211    CNSP-18    CNSP-Inc.com.700
08:56:34$ fm K6RPT-12 to APBL10-0 via WIDE2-1 UI^ PID=F0
          !4051.56N/09355.87WO070/171/A=109987V211    CNSP-18    CNSP-Inc.com.700
08:56:37$ fm K6RPT-12 to APBL10-0 via W0ISU-0,WIDE2-0 UI^ PID=F0
          !4051.56N/09355.87WO070/171/A=109987V211    CNSP-18    CNSP-Inc.com.700
09:02:58$ fm K0SXY-0 to APU25N-0 via WA0ROI-1,W0ISU-0,WIDE2-0 UIv PID=F0
          }W7BOZ-10>APWW10,TCPIP,K0SXY*::K6RPT-12 :?APRSP
09:06:54$ fm K6RPT-12 to APBL10-0 via W0ISU-0,WIDE2-0 UI^ PID=F0
          !4101.42N/09320.35WO071/176/A=110673V211    CNSP-18    CNSP-Inc.com.700
09:24:33$ fm K6RPT-12 to APBL10-0 via WIDE2-1 UI^ PID=F0
          !4118.17N/09214.06WO067/184/A=111368V211    CNSP-18    CNSP-Inc.com.700
09:24:36$ fm K6RPT-12 to APBL10-0 via W0ISU-0,WIDE2-0 UI^ PID=F0
          !4118.17N/09214.06WO067/184/A=111368V211    CNSP-18    CNSP-Inc.com.700
09:26:33$ fm K6RPT-12 to APBL10-0 via WIDE2-1 UI^ PID=F0
          !4113.38N/09206.77WO068/175/A=000108V200    CNSP-18    CNSP-Inc.com.700
09:28:28$ fm N0AN-0 to APU25N-0 via W0ISU-0,WIDE2-1 UI^ PID=F0
          }N8NJ>APWW10,TCPIP,N0AN*::K6RPT-12 :?APRSP
09:28:33$ fm K6RPT-12 to APBL10-0 via WIDE2-1 UI^ PID=F0
          !4122.91N/09158.98WO069/182/A=111736V200    CNSP-18    CNSP-Inc.com.700
09:32:33$ fm K6RPT-12 to APBL10-0 via WIDE2-1 UI^ PID=F0
          !4127.01N/09143.91WO070/181/A=111516V200    CNSP-18    CNSP-Inc.com.700
09:34:33$ fm K6RPT-12 to APBL10-0 via WIDE2-1 UI^ PID=F0
          !4128.87N/09136.29WO072/182/A=111647V200    CNSP-18    CNSP-Inc.com.700
09:34:40$ fm K6RPT-12 to APBL10-0 via W0ISU-0,WIDE2-0 UI^ PID=F0
          !4128.87N/09136.29WO072/182/A=111647V200    CNSP-18    CNSP-Inc.com.700
09:44:36$ fm K6RPT-12 to APBL10-0 via W0ISU-0,WIDE2-0 UI^ PID=F0
          !4138.14N/09057.43WO072/185/A=111277V211    CNSP-18    CNSP-Inc.com.700
09:46:41$ fm K6RPT-12 to APBL10-0 via W0ISU-0,WIDE2-0 UI^ PID=F0
          !4140.04N/09049.68WO072/181/A=111011V200    CNSP-18    CNSP-Inc.com.700
09:52:37$ fm K6RPT-12 to APBL10-0 via W0ISU-0,WIDE2-0 UI^ PID=F0
          !4144.76N/09015.04WO072/198/A=000111V211    CNSP-18    CNSP-Inc.com.700
09:58:49$ fm K6RPT-12 to APBL10-0 via W0ISU-0,WIDE2-0 UI^ PID=F0
          !4149.69N/09001.80WO076/191/A=110889V200    CNSP-18    CNSP-Inc.com.700
10:12:35$ fm K6RPT-12 to APBL10-0 via W0ISU-0,WIDE2-0 UI^ PID=F0
          !4145.27N/08910.26WO080/187/A=000111V200    CNSP-18    CNSP-Inc.com.700
10:15:48$ fm K6RPT-12 to APBL10-0 via W0ISU-0,WIDE2-0 UI^ PID=F0
          !4200.16N/08856.54WO078/182/A=111047V200    CNSP-18    CNSP-Inc.com.700
10:19:41$ fm N0AN-0 to APU25N-0 via W0ISU-0,WIDE2-1 UI^ PID=F0
          }VE3PWD-1>APRS,TCPIP,N0AN*::K6RPT-12 :?APRSP
10:26:35$ fm K6RPT-12 to APBL10-0 via W0ISU-0,WIDE2-0 UI^ PID=F0
          !4154.51N/08812.68WO075/179/A=000111V200    CNSP-18    CNSP-Inc.com.700
10:34:37$ fm K6RPT-12 to APBL10-0 via W0ISU-0,WIDE2-0 UI^ PID=F0
          !4159.33N/08741.31WO078/171/A=000114V211    CNSP-18    CNSP-Inc.com.700
10:36:39$ fm K6RPT-12 to APBL10-0 via W0ISU-0,WIDE2-0 UI^ PID=F0
          !4216.55N/08732.04WO075/168/A=109908V211    CNSP-18    CNSP-Inc.com.700
10:51:53$ fm K0SXY-0 to APU25N-0 via W0ISU-0,WIDE2-0 UIv PID=F0
          }DL2SCH>APRS,TCPIP,K0SXY*::K6RPT-12 :?APRSP

Using Linux to Find Bad Sector Data

Most of us have probably encountered the dreaded NTFS monster.  It's usually easy to fix with chkdsk /f /r or chkntfs.  However, there is one thing that these utilities can NOT fix: bad sectors.

Enter Linux.

If you have a Windows boot disk or WinRE partition, there's a good chance that you can restore a system to full working order if your hard drive doesn't have bad sectors.  However, if you start seeing things like this during a chkdsk:

...then there's a very good chance that you're looking at some hardware level corruption.  In addition to the Blue Screen Of Death (BSOD), I noticed that plugging the HDD into another computer generated a S.M.A.R.T. error!  I could still access data on the drive by mounting it in Linux:

If you don't have the ability to just arbitrarily connect a random HDD to your computer, you could always opt to create a backup over the network.  To accomplish this, Clonezilla is a solid contender.  But, to do it over the network, you'll need a method.  PartedMagic comes out on top by packaging Clonezilla, along with a variety of backup and restore methods.  There are a number of ways you can backup and restore on the network: SSH, CIFS, NFS are just a few to name.  I prefer SSH since it's secure connection is bar-none and the encryption overhead doesn't drag the speed of the connection down too far.  If I was on a firewalled LAN, I'd probably go with CIFS.  However, SSH is ideal for internet-based backup and restores.  It also plays nice with Windows and BSD systems.

Of course, with all backups comes with it the risk of data corruption:

So, what to do?  Well, that's why I have decided to open my case up and hook it up to a free SATA port.  Thankfully, I'm not watching any movies at the moment, so I can disconnect my DVD drive and hook it up that way:

This allows the aforementioned ability to directly mount it.  Now, with that setup, let's see if we can actually find out WHAT files are sitting on WHICH bad sectors, shall we?  First of all, let's see what there is to work with and see what's actually taking up space.  Then, let's look at some important directories:

Hmmm, well the Boot directory looks okay.  The others seem to be doing okay up through MSOCache.  What about ProgramData?

OH!  What's this?  Core dumped?!  When was the last time you saw the du command dump a core?  And did it flush...?  Well, there seems to be some kind of irony to all of this... There appears to be 9 of the original 11 reported bad sectors here.  That means that there are 2 more bad sectors elsewhere on the disk.  I think it's safe to say that the restoration process won't be losing any seriously important data this time around!  Of course, there's more than one way to break a window:

After manually checking the rest of the disk, I could not find the other 2 bad sectors.  It's entirely possible that they were remapped at some point.  Perhaps the failing of S.M.A.R.T. had something to do with it.  Perhaps I just don't know.  That's the risk of backup and recovery, you just never know what's going to happen.  In all likelihood, they are probably buried in the ProgramData directory, but du bottomed out before it got to them.  It would take some digging around to find out the exact spots... that's really beyond the scope of this article.

Of course, if you really wanted to get fancy, you could install smartmontools and get some diagnostic information straight from the drive:

And if you ask it REALLY nicely, it will give you some human-readable content:

Whoops, less than 24 hours left!  I better get a move on...

Addendum: It should be noted that, if you had bad sectors but the S.M.A.R.T. tools says that your disk is fine, then perhaps you should look into shrinking the NTFS partition(s) to avoid the bad sectors.

Welcome!

Welcome to my first blog post!

I know, it's a shocker.  Of all of the social media blogs that I've had, in addition to my own domains (may they R.I.P.), why bother with another?

Well, as you can see, this is my tech blog.  Basically, anything that I do that relates to technology that I wish to share with the world will simply go here.  Howtos, ideas, methods, stuff that I'm working on... it'll all go here.  Computers, electronics, radios, all that fun sort of stuff.  Here.

As this is the first time I've used a blog site that creates everything for me and I just have to add data to it by typing it or adding links and other things, I'm going to go ahead and say that, so far, I like Blogger.  The fact that Google acquired Blogger awhile ago wasn't too surprising, but since they've gone and integrated it into everything else (Gmail, G+, Youtube, Maps, Drive, Calendar, etc etc etc...), I felt that the time came that I wanted to write more than a quick update to G+ or, God forbid, Facebook.  While FB makes me feel things that I wish I couldn't feel, it has a certain usefulness to itself.  Myspace was an interesting theory, as was Yahoo! 360, but it was certainly time to move up and move on.

So here, and now, this blog shall encompass all about me that is geeky, nerdy or otherwise just downright technical.

I won't promise how often I'll update and I won't promise what sort of trends I'll go along.  However, I CAN tell you what my current interests revolve around inside of my head:

• Computers.  Mostly hardware.  I was into computers for over a decade, taking them apart, cleaning them, putting them back together, adding new parts, making them run faster... that sort of thing.  I did the software thing too, but not to the level of engineering degree that I was able to do with hardware.  Eventually, I went on to get an Associates Degree in Information Technology.  So yeah, I self-educated and then went after the real deal.  I suppose you could say I'm an oddball that way.  Yes, yes I am, and damn proud of it.  I worked more than a handful of paid jobs using such insight and had fun at each and every one of them.  I met a lot of interesting people along the way and learned a lot of things about life while I was at it.  It was worth it.
• Radio.  So, eventually, I got my Ham Radio license.  My callsign is KD0FHS.  I found the unique randomness of having such a callsign assigned to me by a computer very difficult to fathom.  Honestly, it was totally random, but to have the suffix, FHS, match up as the same initials of the high school that I attended, Fairport High School, I began to wonder if anyone back there would wonder if I went out of my to get it as a vanity call.  Well, no one's ever really said anything about it, although I have seen a few eyebrows arched.  Well, enough about that, my interest in radio frequencies and other such topics is quite varied.  I enjoy local communication as well as long-range HF stuff.  The ability to talk to and listen to satellites, along with the International Space Station is among some of the fun things that I've always aspired to do.  Being a Ham allows me to explore these things.  Of course, the people that I've met that have taken the time to help me along the way are some of the nicest people I've ever met.  The Amateur Radio community is quite eclectic, but without each other, there wouldn't be a community!
• Electronics.  My interest in general electronics dates back to the late 90's when I got into CB radio.  There really wasn't much to it, but it was fun to tinker around with flashlights and other basic devices.  I even took a class in basic electronics.  Sadly, my interests gravitated toward computers and their operation and less on the electronic side of things.  It would be nearly a decade until I realized the importance of electronics.  That is why I am pursuing my Bachelor's Degree in Electrical Engineering.
• High Altitude Balloons.  Even before I arrived at Uni, I knew that I would never realize my childhood dream of becoming an astronaut.  However, it was many years until I realized that I could help others achieve that dream.  Even if I could not, what right would I have to prevent others from doing so?  There is so much in our own Solar System to explore, and it's just something that I know will happen.  It's already happening with unmanned missions, but in order to test theories and find new facts, it's helpful to go up, WAY up!

So, that's pretty much it in a nutshell.  Right now, I'm finishing up my mathematics and about to delve into the wonderful world of physics.  I have discovered that LaPlace Transforms are easier than I thought they could be.  So, sometimes I might even blog about math.

-Scott