Tag Archives: Errors

Data at the root level is invalid. Line 1, position 1.

Recently, I encountered a really weird problem with an XML document. I was trying to load a document from a string:

var doc = XDocument.parse(someString);

I received this unhelpful exception message:

Data at the root level is invalid. Line 1, position 1.

I verified the XML document and retried two or three times with and without the XML declaration (both of which should work with XDocument). Nothing helped, so I googled for an answer. I found the following answer on StackOverflow by James Brankin:

I eventually figured out there was a byte mark exception and removed it using this code:

string _byteOrderMarkUtf8 = Encoding.UTF8.GetString(Encoding.UTF8.GetPreamble());
if (xml.StartsWith(_byteOrderMarkUtf8))
{
    xml = xml.Remove(0, _byteOrderMarkUtf8.Length);
}

This solution worked. I was happy. I discussed it with a coworker and he had never heard of a BOM character before, so I thought “I should blog about this”.

Byte-Order Mark

The BOM is the character returned by Encoding.UTF8.GetPreamble(). Microsoft’s documentation explains:

The Unicode byte order mark (BOM) is serialized as follows (in hexadecimal):

  • UTF-8: EF BB BF
  • UTF-16 big endian byte order: FE FF
  • UTF-16 little endian byte order: FF FE
  • UTF-32 big endian byte order: 00 00 FE FF
  • UTF-32 little endian byte order: FF FE 00 00

Converting these bytes to a string (Encoding.UTF8.GetString) allows us to check if the xml string starts with the BOM or not. The code then removes that BOM from the xml string.

A BOM is a bunch of characters, so what? What does it do?

From Wikipedia:

The byte order mark (BOM) is a Unicode character used to signal the endianness (byte order) of a text file or stream. It is encoded at U+FEFF byte order mark (BOM). BOM use is optional, and, if used, should appear at the start of the text stream. Beyond its specific use as a byte-order indicator, the BOM character may also indicate which of the several Unicode representations the text is encoded in.

This explanation is better than the explanation from Microsoft. The BOM is (1) an indicator that a stream of bytes is Unicode and (2) a reference to the endianess of the encoding. UTF8 is agnostic of endianness (reference), so the fact that the BOM is there and causing problems in C# code is annoying. I didn’t research why the UTF8 BOM wasn’t stripped from the string (XML is coming directly from SQL Server).

What is ‘endianness’?

Text is a string of bytes, where one or more bytes represents a single character. When text is transferred from one medium to another (from a flash drive to a hard drive, across the internet, between web services, etc.), it is transferred as stream of bytes. Not all machines understand bytes in the same way, though. Some machines are ‘little-endian’ and some are ‘big-endian’.

Wikipedia explains the etymology of ‘endianness’:

In 1726, Jonathan Swift described in his satirical novel Gulliver’s Travels tensions in Lilliput and Blefuscu: whereas royal edict in Lilliput requires cracking open one’s soft-boiled egg at the small end, inhabitants of the rival kingdom of Blefuscu crack theirs at the big end (giving them the moniker Big-endians).

For text encoding, ‘endianness’ simply means ‘which end goes first into memory’. Think of this as a direction for a set of bytes. The word ‘Example’ can be represented by the following bytes (example taken from StackOverflow):

45 78 61 6d 70 6c 65

‘Big Endian’ means the first bytes go first into memory:

45 78 61 6d 70 6c 65
<-------------------

‘Little Endian’ means the text goes into memory with the small-end first:

45 78 61 6d 70 6c 65
------------------->

So, when ‘Example’ is transferred as ‘Big-Endian’, it looks exactly as the bytes in the above examples:

45 78 61 6d 70 6c 65

But, when it’s transferred in ‘Little Endian’, it looks like this:

65 6c 70 6d 61 78 45

Users of digital technologies don’t need to care about this, as long as they see ‘Example’ where they should see ‘Example’. Many engineers don’t need to worry about endianness because it is abstracted away by many frameworks to the point of only needing to know which type of encoding (UTF8 vs UTF16, for example). If you’re into network communications or dabbling in device programming, you’ll almost definitely need to be aware of endianness.

In fact, the endianness of text isn’t constrained by the system interacting with the text. You can work on a Big Endian operating system and install VoIP software that transmits Little Endian data. Understanding endianness also makes you cool.

Summary

I don’t have any code to accompany this post, but I hope the discussion of BOM and endianness made for a great read!

Flattr this!

System76 notebooks… are they worth the money?

On June 19 2012, I purchased a 15″ Pangolin Performance with the following configuration:

  • Ubuntu 12.04 LTS 64 bit
  • 15.6″ 720p High Definition LED Backlit Display ( 1366 x 768 )
  • Intel HD Graphics 4000
  • 3rd Generation Intel Core i7-3610QM Processor ( 2.30GHz 6MB L3 Cache – 4 Cores plus Hyperthreading )
  • 16 GB Dual Channel DDR3 SDRAM at 1600MHz – 2 X 8 GB
  • 750 GB 7200 RPM SATA II
  • 8X DVD±R/RW/4X +DL Super-Multi Drive
  • Intel Centrino 1030 – 802.11 b/g/n Wireless LAN + Bluetooth Combo Module

Total price: $1,038.00

The price was higher than I wanted to spend, but I considered this purchase an investment. After all, I had previously owned a Sony Vaio for almost 5 years, and which I had purchased for $800. The reviews for System76 seemed a bit mixed. I’ve been using Linux or BSD as my main desktop environment since 1998, so I took a chance on the mixed reviews and made the investment.

Issue #1

After two months, I had to open a ticket because the Ubuntu version preinstalled on the machine displayed the graphics card as ‘Unknown’ and the screen itself exhibited a sort of wave of energy which was giving me headaches. I was told by the support engineer that the fix would be to install the mesa-utils package. Why is mesa-utils not installed by default if it’s a necessary package? I have no idea, but here’s the response I received from my support ticket:

It’s actually a known issue without having the mesa-utils installed. Apparently, the info is actually pulled from glxinfo.

Mesa-Utils isn’t part of the default install, and admittedly, we do not add it either. I know there’s talk of trying to add it as part of the install.

Sure.

The graphics issues were actually resolved by an early upgrade to Ubuntu 13.04.

Issue #2

In October 2012, I decide to dual boot a copy of Windows 8 from MSDN. There were numerous issues with drivers. After opening a support ticket, I was told that System76 doesn’t have access to drivers for their hardware from their vendors (bison or chicony hardware). That seems really shady, considering a machine for such a high price should have quality hardware. I ended up removing Windows 8 and installing Windows 7 within a VirtualBox VM. The hardware is beefy enough to host multiple concurrent VMs, so this is actually my preferred method for cross-platform development. However, a machine that can’t dual boot Windows basically renders the machine useless for developers who need to code at a low level like packet processing or driver development.

Issue #3

On June 15 2014, just shy of two years after purchasing this machine, I decided to open a ticket regarding battery issues I started having after installing Ubuntu 14.04. My battery began to hold no more than 75% or so charge. I inquired how to troubleshoot whether or not there were some other factors causing battery issues. For reference, here are screenshots of the battery statistics:

I was told by the support team that my battery was causing problems because:

The statistics indicated this is within your battery. The energy designed (48.8 W/hr) and full (40.0 W/hr) is where your discrepancy lies.

So, I opened a ticket saying that my machine had a max of 76% charge after hours of charging (both booted and off) and support replied saying my problem was that my battery could only hold a max charge of 83.3%. The resolution offered was to purchase a new battery at $105.

That’s not right. Every two years of ownership, I will need to purchase the same low quality battery at $105 out of pocket? In a sense, I have purchased a machine that also requires a $50/year battery fee. This seems really silly.

I admit that I had no battery issues for about two years. In fact, I was surprised by the battery life (2.5 to 3 hours) early on. Machines with a mechanical hard drive and 16GB of RAM are generally considered battery hogs. I don’t know of anyone who actually works on such a machine on battery.

Conclusion

Although my machine has worked well for two years, I don’t like the quality of the hardware or the poor response from the company about the issues I have had. I probably wouldn’t mind it if the quality of support offset the quality of the hardware. For example, if System76 actually stood behind their product and replied “Wow, our batteries should last more than 1.95 years. We’ll send you a replacement immediately,” I would recommend purchasing from the company. You do get a pretty beefy machine for relatively cheap. As a software engineer with multiple side projects and a 9 month old son, I don’t have time to continuously try to troubleshoot issues that I really shouldn’t be having with a quality piece of hardware.

My own resolution was to purchase a 13″ Macbook Pro. This machine has 8GB RAM and a 256GB SSD Hard Drive. I have been getting 8 or 9 hours of battery life, and that’s with the power-sucking Google Chrome open at all times. I became accustomed to the OS X environment while working at Expedia. Mac OS X is a UNIX certified OS, so I feel at home in the environment. The only issue I had with OS X when I first started working in the environment was the difference between COMMAND, OPTION, and CONTROL. This took a week or two to become habit. My wife also left her Sony VAIO for a Macbook Air last year and she loves it. You’d be hard-pressed to find a Linux distribution which allows for such a smooth computing experience for non-geeks.

I will be selling my System76 machine after backing up all important information. If you’re interested, let me know. You’d be getting a beefy machine and not taking along any of the annoyance of paying over $1000 for subpar customer support.

To answer the question “Are they worth it?”, I would have to say “It depends.” If you want a machine that would cost $2500-3000 elsewhere for around $1000, then yes it is definitely worth it. If you don’t mind replacing a battery after two years, then yes it’s worth it. If you’ve used Linux for a long time like I have and don’t mind spending an hour or two every time a ‘surprise’ surfaces, then yes it’s definitely worth it. If you’re like me and you have little free time, then you’ll want to open your machine and expect everything to work as expected with little or no interaction with customer support. In my case, it’s just not worth it to own a machine that requires so much maintenance. If this system was a car, I would sell it as a car that runs well and needs little or no work. That is, if you don’t work off battery you’d be all set.

Flattr this!

HostGator’s upgrade to PHP 5.4

Recently, HostGator sent around some emails saying they were forcing a PHP upgrade to 5.4 in the coming weeks. The steps provided by HostGator are basically:

  1. Login to CPanel
  2. Go to PHP Configuration
  3. Change from 5.2 to 5.4
  4. See if your site works

I’m off work this week to work on my self-published book, so I went ahead and tried the PHP 5.4 upgrade. First, I made sure my sites were backed up and all WordPress plugins/themes were updated.

After running the PHP 5.4 configuration change, my site didn’t work. I received an HTTP 500 error with the cute alligator smiling his smug face at me. I’m on the current version of WordPress with everything (including my database) fully up-to-date. Naturally, the first thing I did was check the error logs in CPanel… empty. Next, I contacted customer support.

I was on with support for 1h 15m. At first, the support representative couldn’t see the internal server error page. She could run through kproxy.com and see my main page. I tried kproxy.com and saw my site, while other proxies and is-it-down-type checkers reported my site as being down.

After some time, we tracked down the following error message:

Cannot load the ionCube PHP Loader - it was built with configuration 2.2.0, whereas running engine is...

I was initially told this was a problem with my theme (which is the default theme with WordPress, btw). I was getting frustrated, but with the error message I was able to Google what would cause this and it turned out to be a php.ini issue. At first, I assumed it was the server’s php.ini causing problems. After about 5-10 minutes of the support representative investigating further, I loaded ~/www/php.ini from within my SSH session and found that the PHP configuration switcher in HostGator’s CPanel doesn’t modify php.ini in any way to account for the updated PHP version.

zend_extension="/usr/local/IonCube/ioncube_loader_lin_5.2.so"

The fix for this whole problem was to change the version of IonCube used for script loading to match the version of PHP I chose in the PHP Configuration switcher. That’s really something that should be done automatically by a utility. Without error logs displaying in CPanel, I don’t know how the average customer is supposed to figure that one out.

Even after the day of downtime recently, I still give HostGator a 9/10. They’re always quick to respond to customer concerns and very nice. They resolve issues as quickly as possible and provide a pretty solid and affordable hosting environment.

Flattr this!

Install Balsamiq, Ubuntu 14.04

Balsamiq for Desktop requires Adobe AIR, which is no longer supported.

Here are the steps I took to get AIR and Balsamiq to install.

gnome-keyring is required by AIR, gdebi is required to install the balsamiq package which will fail as “bad quality” in the Ubuntu Software Center. You can skip the gdebi install if you’re using X File Package (or probably anything other than Ubuntu Software Center) to install .deb packages.

sudo apt-get install gnome-keyring gdebi gdebi-core

Lots of sites say you need to install ia32-libs, which don’t exist in the Ubuntu repositories. Here is what’s required (taken from this post on askubuntu.com and this other post on askubuntu.com )

sudo apt-get install gtk2-engines:i386 libart-2.0-2:i386 libcairo2:i386 libcanberra-gtk0:i386 libdatrie1:i386 libgail-common:i386 libgconf2-4:i386 libgtk2.0-0:i386 liblua5.1-0:i386 libpango1.0-0:i386 libpixman-1-0:i386 libqt4-network:i386 libqt4-test:i386 libqtcore4:i386 libthai0:i386 libbonobo2-0:i386 libglade2-0:i386 libgnomecanvas2-0:i386 libidl0:i386 liborbit2:i386 libwmf0.2-7:i386 gtk2-engines-murrine:i386 libxml2:i386 libxslt1.1:i386 libxt6:i386 lib32nss-mdns libnspr4-0d:i386 libnss3-1d:i386 

Now, download AdobeAIRInstaller.bin version 2.6 and MockupsForDesktop64bit.deb. (If MockupsForDesktop64bit.deb link is dead, please download from here.)

cd ~/Downloads
chmod +x AdobeAIRInstaller.bin # make Adobe AIR executable
LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu ./AdobeAIRInstaller.bin # install Adobe AIR

Once AIR is installed, open your file manager, right-click on the balsamiq package and choose open with -> gdebi. The install should now complete.

Flattr this!

Android Studio and Library Projects

This is basically a quick brain-dump post. I have previously attempted to get into Android application development, but with only 8-10 hours of “free” time per month, it was difficult to get traction with an app before the goog machine overhauled everything. This happened repeatedly. I had some great ideas for a mobile app over the weekend, and thought I’d give it another go and guess what? Everything is overhauled: Android Studio is the next big thing, ant builds are out and gradle builds are in. I decided to force myself to overcome my annoyances and try out Android Studio.

Android Studio is beautiful and operationally stable, although very buggy. I’ve found four bugs in less than two days. The biggest bug and usability issue, however, is in creating an Android Library project and adding a reference to it in another project.

Here are the steps I took to reference another project. These steps may not be accurate, but I don’t care. I figured they may help save someone some time.

  1. Create an Android library project using Android Studio.

    There’s a bug in Android Studio 0.1.3 which apparently does not mark android library projects as library projects, so navigate to MyLibProject/MyLib and change the android plugin reference to:

    apply plugin: 'android-library'
  2. Create a main application project, in my case MyApplication2
  3. Add library as git submodule or nested directory under MyApplication2
  4. Edit settings.gradle in the main project to:
    include ':MyLibProject:MyLib’, ':MyApplication2'
  5. Edit MyApplication2/build.gradle to compile the lib project in the dependencies task:
    compile project(':MyLibProject:MyLib’)
  6. Navigate to your library subdirectory and execute:
    gradle clean && gradle assemble
  7. Press CTRL+ALT+SHIFT+S to open the Project Structure dialog
  8. Create a new module, change module name and point content root to MyLibProject
  9. Change Package name to the package name of your lib and press Finish
  10. Click MyApplication2 (not MyApplication2Project) in the Project Structure dialog and select the Dependencies tab.
  11. Click the green plus icon and select Library|Java
  12. Select MyLibProject/MyLib/build/bundle/release folder, choose Project Library, and hit ok
  13. Save. The library should now be usable.

These instructions may seem a bit hurried, but it should get the job done. I’ve run through numerous attempts at different options, and these are the only ones that seem to have stuck.

I might also mention, I’ve created an ANDROID_HOME environment variable to load in my shell which points to the sdk directory under the android-studio installation. I’ve also downloaded gradle-1.6 to ~/bin, and symlinked gradle to ~/bin/gradle which adds gradle to my path.

Flattr this!

“Unable to update the dependencies of the project”

I’ve recently had to switch from Visual Studio 2012 back to Visual Studio 2010 to do maintenance on another project. This project is currently stuck in Visual Studio 2010 until I have time to convert the old setup projects to WiX. I often receive the following error message during Release builds:

"Unable to update the dependencies of the project"

Closing Visual Studio 2010 and reopening seemed to have fixed the problem up until about a week ago. I’ve found that installing this hotfix seems to resolve the issue. The only problem is that I’ve had to install the hotfix multiple times.

The ’cause’ on the hotfix page says the issue is a result of how Visual Studio 2010 refreshes dependencies. I wonder if this is handled by Windows Updates updating the .NET Framework? Whatever it is, it’s pretty annoying to have to apply this patch regularly. I guess it’s just another reason for developers to move setup projects to WiX.

http://support.microsoft.com/kb/2286556

Flattr this!

git push: fatal: unable to read SHA1

Today, I was faced with an interesting error in a git repository. I am backing up a lot of old projects from during and after college into a private git repo. In doing so, I moved some folders around which disconnected a couple of binary files. After pushing, I received an error: unable to read [SHA1].

The fixes, in short:

$ git fsck
$ git log --raw --all --full-history | grep SHA1-HERE
$ git hash-object -w OBJECT-PATH-HERE
$ git push

Here is the error and a walk-through of coming up with the fixes above:

jim at schubert in /media/16GB/projects/school on master
$ git push
Password: 
Counting objects: 1945, done.
error: unable to find 2978ec4d75abb8c6bab225d8adfbd2bef064338a
error: unable to unpack bddbd13afd698e5ba7d572c9270e52bcac862661 header
error: inflateEnd: failed
Delta compression using up to 2 threads.
Compressing objects: 100% (1854/1854), done.
fatal: unable to read 2978ec4d75abb8c6bab225d8adfbd2bef064338a
fatal: The remote end hung up unexpectedly
fatal: The remote end hung up unexpectedly
fatal: write error: Bad file descriptor

After running git fsck, I found that I had two missing blobs:

jim at schubert in /media/16GB/projects/school on master*
$ git fsck 
dangling tree dbe9172996edbb7df517b0305c38891d78b72f66
dangling tree fbf7d8336b5f2347da23eb8a3938de5ab18f783c
missing blob 2978ec4d75abb8c6bab225d8adfbd2bef064338a
missing blob bddbd13afd698e5ba7d572c9270e52bcac862661

To fix this, I had to get the filenames of these blobs and write them back into the repository:

jim at schubert in /media/16GB/projects/school on master*
$ git log --raw --all --full-history | grep bddbd13
:000000 100644 0000000... bddbd13... A	INFO 465/Project2/UseCase/Diagrams/Leader - Time & Mileage.vsd

jim at schubert in /media/16GB/projects/school on master*
$ git log --raw --all --full-history | grep 2978ec4
:000000 100644 0000000... 2978ec4... A	INFO 465/Project2/Prototype/WebPrototype/WebPrototype/bin/WebPrototype.dll

Writing these files back into the repository, the push was successful. To write these back, do the following:

jim at schubert in /media/16GB/projects/school on master
$ git hash-object -w INFO\ 465/Project2/UseCase/Diagrams/Leader\ -\ Time\ \&\ Mileage.vsd
bddbd13afd698e5ba7d572c9270e52bcac862661
jim at schubert in /media/16GB/projects/school on master
$ git hash-object -w INFO\ 465/Project2/Prototype/WebPrototype/WebPrototype/bin/WebPrototype.dll
2978ec4d75abb8c6bab225d8adfbd2bef064338a

Flattr this!

System.Data.OracleClient and Windows 7… love at first sight!

</sarcasm>

I spent nearly two days trying to resolve this issue. My new desktop at work is running Windows 7 (64-bit) and some of our applications are still using System.Data.OracleClient as an adapter instead of Oracle’s own ODP.NET. Microsoft’s driver interops with Oracle’s own client installed on the developer machine (oci.dll). We need to continue running our web applications under 32-bit IIS, which is where the problem lies.

FYI… According to Microsoft:

The types in System.Data.OracleClient are deprecated. The types are supported in version 4 of the .NET Framework but will be removed in a future release. Microsoft recommends that you use a third-party Oracle provider.

Back to the comment about IIS. Apparently, IIS 7 in Windows 7 is 64-bit only. Yes, you can set an application pool to 32-bit. That would be perfectly fine, except that Oracle’s client native methods don’t like running through WoW64. So, you have to install both the 64-bit client and the 32-bit client. After all, 32-bit adapter code can’t call a 64-bit client. If you fire up procmon, though, you’ll see that calling the 32-bit Oracle client still queries 64-bit settings (and fails if they don’t exist). WoW64!, indeed.

Continue reading to see my resolution…
Note: this forces Oracle 11.2.0.1 to use 32-bit only in a 64-bit environment.
Continue reading System.Data.OracleClient and Windows 7… love at first sight!

Flattr this!

Hostgator: ssh warns ‘too many authentication failures’

I requested for HostGator to provide ssh access. Since then, I haven’t used it. I attempted to ssh into the server today and received the message ‘too many authentication failures’.

If you haven’t done so already, create a PubKey:

ssh-keygen -t dsa

The ssh-keygen program will ask a few questions such as the key name and the passphrase. Enter whatever you’d like. I’ll refer to the file as id_dsa.pub (which is the default name).

Navigate to your .ssh directory:

cd ~/.ssh

Make sure your file exists in this directory.

Now this one-liner does it all:

cat ~/.ssh/id_dsa.pub | ssh -p 2222 -o PubkeyAuthentication=no user@hostname 'cat >> .ssh/authorized_keys'

If you’re not familiar with the linux pipe, I will explain a bit. If you understand the above command, you’re done. Enjoy.

The first part before the vertical line {the ‘|‘ is called a pipe) dumps all of the text (cat) inside the file (~/.ssh/id_dsa.pub) into STDOUT, or the standard output stream. The pipe forces STDOUT from before the pipe to become STDIN (standard input) for the command after the pipe. Normally, on HostGator you would open an ssh session with ssh -p 2222 user@hostname. But, because there is a conflict in either existing keys on your machine or the authentication of those keys on the server, you’ll have to add -o PubkeyAuthentication=no. Finally, the command within single quotes at the end dumps the text in STDIN into a file handle (the ‘>‘ causes the dumping of this text to redirect STDIN to an existing filehandle, while two of those characters ‘>>‘ redirects to a file, creating the file if it doesn’t exist).

Flattr this!

Can’t login to MySQL as root :(

I was having issues logging into MySQL as root the other day. It took a while to figure things out. Ultimately, I had to follow the steps on two separate posts: http://rimuhosting.com/howto/mysqlinstall.jsp and http://bugs.mysql.com/bug.php?id=22118.

Here are the steps I took:

  1. Stop MySQL:
    /etc/init.d/mysql stop
  2. Edit config:
    sudo vim /etc/my.cnf
  3. Add line to section [mysqld]:
    skip-grant-tables
  4. Start MySQL:
    /etc/init.d/mysql start
  5. Reset Root Password:
    mysql -e &quot;update user set password = old_password('newpassword') where user = 'root'&quot; mysql
  6. Kill MySQL:
    kill `cat /var/run/mysqld/mysqld.pid`
  7. Create an init-file:
    cat ~/mysql-init&lt;&lt;EOF
    UPDATE mysql.user SET Grant_priv='Y', Super_priv='Y' WHERE User='root'; 
    FLUSH PRIVILEGES;
    GRANT ALL ON *.* TO 'root'@'localhost';
    EOF
    
  8. Run MySQL with the init file:
    mysqld_safe --init-file=~/mysql-init &amp;
  9. Remove the mysql-init file:
    rm ~/mysql-init
  10. Restart MySQL:
    /etc/init.d/mysql restart
  11. Login as root!

Part of this was also taken from the MySQL documentation for resetting a password, however, the contents of the file were changed to match one of the previously mentioned posts. If these steps don’t work, please visit the official documentation and give it a try again.

**IMPORTANT**
When you’re done following the above steps, please remove ‘skip-grant-tables’ from my.cnf. Thanks, Ian, for bringing this missed step to my attention!

Flattr this!