Category Archives: Sharpen The Saw

Boxing and Unboxing in C#

This post is an aide-memoire as I learn more about boxing and unboxing in C# and is based upon this part of the C# docs.

Boxing

Is the process of converting a value type (such as int or bool) to the type Object or to any interface type implemented by this value type. When the CLR boxes a value type, it wraps the value inside a System.Object and stores it in the managed heap.

Boxing is implicit.

int i = 10; 
// this line boxes i
object o = i;

Although it is possible to perform the boxing explicitly it is not required.

int i = 10;
// explicit boxing
object o = (object)i;

Unboxing

Extracts the value type from the object.

Unboxing is explicit.

int i = 10; 
// boxes i
object o = i;
// unboxes the object to the int value type named j
int j = (int)o;

Performance

Both boxing and Unboxing are computationally expensive operations.

 

C# Utility to emulate the XPath 3 function path()

I recently needed to examine a number of XML files and print out the element names that contained text greater than X number of characters. In addition I also need to print the location of that element within the XML document.

i.e. given the following document

<bookshop>
  <book>
    <title>Microsoft Visual C# Step by Step</title>
  </book>
</bookshop>

…if I was interested in book titles that had more then 10 characters I would want to see:

/bookshop/book/title/
Microsoft Visual C# Step by Step

Whilst it is straightforward to return the text node, finding the XPATH location proved to be more challenging than I initially thought. The reason being is that whilst XPATH 3.0 introduced the path() function that returns the current XPATH location, the number of programming languages that I know (PL/SQL, Python and C#) do not implement XPATH 3.0 yet.

As a result I had to build my own utility. I chose to write this in C# as this is a language I have spent the past 18 months learning and I am now looking for real world problems I can solve using it.

The utility can be found on github. The “engine” of the utility is copied from this Stackoverflow answer: http://stackoverflow.com/a/241291/55640 provided by Jon Skeet.

Although far from feature complete I hope it will give someone facing a similar challenge a head start.

Let me know what you think.

Contributing to an Open Source Project

I have been interested in Git, the distributed version control software since reading the first edition of Scott Chacon’s Git book way back in 2010. However outside of my own projects, my real world experience of using Git is relatively limited and it’s one of the skills I never seem to get around to improving on.

To change this, I have recently contributed to an open source project hosted on GitHub. The change I made can be found here and this post is my recollection of the process to help me and hopefully others just getting started with the GitHub workflow.

For a comprehensive guide to the GitHub workflow, I recommend reading Chapter 6 of the Git book.

Find a project that you want to contribute to.

Probably the most tricky step – there are so many projects how do you find one to contribute to? In my case I started with a project that I know and use. OraOpenSource/Logger which is a great tool for instrumenting Oracle PL/SQL code.

githubissues

 

From there it’s a quick scan of the open issues. I picked one related to the documentation because I wanted to focus on the GitHub workflow. The challenging technical issues and enhancements will still be there once I have got up to speed with the GitHub way of working.

Once you have found a project, it is unlikely that you will be able to push your changes to it, so the next step is to fork it. This gives you a copy of the project within your users namespace which you can then make changes to.

Make the change

With the project forked, you can go ahead, create a topic branch and make the necessary changes to the files and once you are happy with them, push them back to your copy of the project.

Pull request and …..Oops!

When you are ready to contribute your changes back to the original project you need to create a pull request. Creating a pull request opens up a discussion thread with a code review focusing on your proposed change.

Don’t worry if the change is discussed or rejected. Dust yourself down and go again. I had my own oops moment with my first pull request as I had changed a URL from relative to absolute. Not a problem so I closed the initial pull request and created another which has now been accepted and merged into the project.

Make the world a better place

Apologies the for the heading, I have been enjoying Silicon Valley around the time this post was taking shape.  If not the world, your change no matter how small will make the project you are contributing to better and it gives you a public artefact that you can point to.

Summary

In this article I had written about my Git experience along with my first contribution to an open source project.

 

A step by step guide to building a Raspberry Pi Hedgehog camera

 

What the camera was designed to do! Photograph of a Hedgehog in the garden

I had been looking for an Raspberry Pi project for a long time, the various media player and game emulators didn’t interest me enough besides I wanted something that the whole family could get involved in.  It was only through discussing this at work that someone mentioned they had built a Hedgehog camera with a Raspberry Pi. Awesome! That was exactly the project I was looking for.

So over the course of several nights I built a prototype and was genuinely surprised at a) how much fun I had putting it together, b) that it actually worked and finally c) how much the whole family has got involved.

One thing I found pretty much straight away was although there is plenty of information on the web to build one of these devices it is not all in one place and I found myself going from one site to another so this post is my attempt to bring all the information and what I learnt whilst building it into one place. I will of course acknowledge which sites helped.

Before going any further here is what my device looks like.

Ugly but effective! The completed Hedgehog camera

 

Granted, it won’t win any design awards. You can spend more time making it look far more aesthetically pleasing if you so wish.

What you will need and how much it will cost

A Raspberry PI (I am using the Raspberry Pi 3 and bought this starter kit)£41.50
A Raspberry PI Case. Unfortunately the official case that comes with the kit above requires you to remove the top part in order to fit the camera and as the PI will be spending a lot of it’s time outside I wanted to protect it as much as I could so I bought this case which has a slot for the camera ribbon£4.50
Waveshare Raspberry Pi Camera Night Vision Camera Module Kit£23.99
An air tight storage box.From 99p
A drill bit to make a hole in the storage box for the camera. After managing to destroy several boxes by using the wrong tools, I ordered this drill bit set which made the task a breeze.£7.95
A Battery Pack. I use this one but at the time of this post the cost has gone up considerably since I bought mine. You just need one that has a large capacity, the one I am using has an advertised capacity of 20100mAh and comfortably powers the camera all night.Around £25 - £30
Total:£109.44

Setting up the Raspberry Pi

If you are using an existing Raspberry PI and its software is up to date , then you can skip this step.  Out of the box my Raspberry PI required various updates. Using the advice from here  I ran

sudo apt-get update

to update the package list followed by

sudo apt-get dist-upgrade

to upgrade all packages to their latest version.

If you are going to use the default user, Pi remember to change the password!

Fitting the camera and installing the camera software

It is important to note that the Waveshare camera is an all in one device. By which I mean it comes with everything needed to take pictures at night time. You don’t need extra infra red led’s or motion sensors. I should also point out that the camera takes stills and not videos. What you can do using the software that I mention below is to “stitch” all the pictures taken in one night together to create a time lapse MP4 file that you can watch, upload to YouTube etc.

First you need to fit the camera to the Raspberry Pi. This video is an excellent tutorial, even though the camera module in the video is different, the installation is exactly the same.

With the camera now fitted you need to install software to control the camera and when pictures are taken. I chose pi-timolo.  This excellent piece of software can be configured to take photo’s whenever motion is detected. The installation is straight forward and as my Raspberry Pi is only being used for the Hedgehog camera I also followed the instructions to run on boot. Which, in case of need means that when the Raspberry Pi starts up, the camera starts capturing images once it detects motion.

The default location for the photos are:

/home/&lt;&lt;username&gt;&gt;/pi-timolo/motion

where <<username>> is the name of the user that installed pi-timolo.

At this point, we now have a Raspberry Pi camera that will take a photograph whenever it detects motion which means that the number of photographs taken every night will be measured in the hundreds. Sifting through these every day will very quickly lose it’s appeal (if it ever had one) so the final software package we are going to install is FFmpeg. This will “stitch” all the photographs for a day (or folder) into one MP4 file. Playing the MP4 file will give you a time lapse like experience. You can download FFmpeg from here. I will show you how to use FFmpeg in the Reviewing the Results section.

Fitting the PI and camera into the air tight container

Overview of the components within the storage box

Firstly make a guide in the middle of the airtight container. This should be at one end of container. Then using the drill bit I have suggested or another tool drill\cut a circle so that the camera can be fitted. You do not need to cut similar holes for the infra red sensors.  Then fix the camera to the inside of the container. I have used parcel tape(!) to secure mine. Not elegant but it works.

If you have not done so, now is the time to connect the camera to the Raspberry Pi.

The final step is to connect the battery to the Raspberry Pi, I normally do this just before I place the container in the garden.

The parcel tape around the outside of the box is to cut down the amount of light that is visible from within the box.

Where to place in the garden

The obvious answer is to place the camera in the area where you have seen Hedgehogs or their poo! However the key is to experiment so try various places around your garden.

Reviewing the Results

After collecting the container the following morning,  my workflow for reviewing the results is to first power down the Raspberry Pi (it is still using the battery at this point) and then connect it to the mains and restart it.

After navigating to

/home/&lt;&lt;username&gt;&gt;/pi-timolo/motion

I copy the photos to another folder within my home directory called

/home/&lt;&lt;username&gt;&gt;/garden/&lt;&lt;date_of_photos&gt;&gt;

and using the following FFmpeg command

ffmpeg -f image2 -pattern_type glob -i '*.jpg' 20160821.mp4

which creates an MP4 file from all the jpg files within the directory where this command was run.

After a few minutes the process will be complete and you will be able to play the MP4 file using your favourite player. When viewing on the Raspberry Pi I use Omxplayer  or other devices I use VLC.

Optional Extras

One optional extra that springs to mind is to add a WiFi extender dongle (if your garden is large enough) this will enable you to connect to the device and see what is happening in real time.

If you think of other optional extras please let me know via the comments.

Summary

In this article I have gone through the steps required to create a Hedgehog camera using a Raspberry Pi.

This write up has been made a couple of weeks after I had built my own version, so if I have missed something or I haven’t made a particular point clear enough please let me know in the comments.

Acknowledgements

My gratitude for a large part of this work, in particular the genius of using the air tight container goes to http://www.sconemad.com/blog/hedgeycam/

 

 

Raspberry Pi connected to WiFi but no internet access

A quick tip in case you run into the following problem as I did with a Raspberry Pi 3 running the Jessie version of Raspbian.

Problem

Unable to access the Internet using the Raspberry Pi’s built in WiFi.

Symptoms

  1. You can successfully connect to the WiFi network
  2. You are unable to access any Internet sites.
  3. Running the command route -n in a terminal shows a default gateway of 0.0.0.0
  4. Within the file /etc/network/interfaces you see the following:

    iface wlan0 inet manual
    wpa-conf /etc/wpa_supplicant/wpa_supplican.conf

Resolution

Using your favourite text open the file /etc/network/interfaces and find the following line:

iface wlan0 inet manual
wpa-conf /etc/wpa_supplicant/wpa_supplican.conf

Amend or comment out the line iface wlan0 inet manual and change it to:

iface wlan0 inet dhcp
wpa-conf /etc/wpa_supplicant/wpa_supplican.conf

The only change is shown in bold i.e manual has changed to dhcp. The screen shot below shows how the interfaces file looks with the original manual entry commented out and the corrected line with dhcp added:

interfaces file

Reboot your Raspberry Pi, run the command route -n and you should see a new line with the
gateway details once confirmed open your Browser and you should now be able to connect to the Internet.

What is DHCP?

The following is taken from here.

Dynamic Host Configuration Protocol (DHCP) is a network protocol that enables a server to automatically assign an IP address to a computer….

Using INSERT ALL with related tables

In this article I will demonstrate a method where the INSERTALL statement can be used to insert rows into two tables that are related by a foreign key constraint.

The following example was developed and tested using the Pre-Built Oracle Database App Development VM which at the time of this post used Oracle Database 12.1.0.2.0  & SQL Developer 4.0.3.16.  The example was created within the standard HR schema.

First I create two tables:

CREATE TABLE parent_tbl (the_pk    NUMBER PRIMARY KEY,
                         object_id NUMBER);

CREATE TABLE child_tbl (the_pk      NUMBER PRIMARY KEY,
                        the_fk      NUMBER,
                        object_name VARCHAR2(30));

Next I add a foreign to define the relationship between these tables and two sequences used to populate the primary keys:

ALTER TABLE child_tbl
ADD CONSTRAINT child_tbl_fk1 FOREIGN KEY (the_fk)
   REFERENCES parent_tbl (the_pk);

CREATE SEQUENCE parent_tbl_seq;

CREATE SEQUENCE child_tbl_seq START WITH 500;

Next I ensure that the foreign key relationship is working as expected by trying to insert a record into the child table with value for the_fk column that doesn’t exist in parent_tbl:

INSERT INTO child_tbl
(the_pk,
the_fk,
object_name)
VALUES
(child_tbl_seq.nextval,
999,
'SomeObject');

Attempting to run this statement results in the expected error message:

SQL Error: ORA-02291: integrity constraint (HR.CHILD_TBL_FK1) violated - parent key not found

With the tables and relationship between them in place I can now demostrate how to use INSERTALL to insert information from user_objects into the parent and child tables.

INSERT ALL
INTO parent_tbl(the_pk, object_id) 
VALUES (parent_tbl_seq.nextval, 
        object_id)
INTO child_tbl(the_pk, the_fk, object_name) 
VALUES (child_tbl_seq.nextval,
        parent_tbl_seq.currval,
        object_name)
SELECT uo.object_id,
       uo.object_name
FROM user_objects uo
/

Lines 2 – 4 insert into the parent table, note the use of the sequence to populate the primary key.

Lines 5 – 8 insert into the child table, the important part (and the focus of this article) is to understand how the foreign key column, the_fk is populated. It uses currval of the sequence that was used to populate the parent table.

Thanks go to Tom Kyte for coming up with this elegant solution.

Summary

In this article I have demonstrated how to use INSERTALL to insert into two tables that are related by a foreign key.

Acknowledgements

The idea for this article came from Tom Kyte’s answer to this Ask Tom question.

The best lesson I learnt from Steve McConnell

CodeCompleteI was made aware of the book Code Complete by Steve McConnell almost by accident. I was attending an Oracle presentation back in 2001 and the presenter, Steven Feuerstein mentioned the book. I can’t remember what context it was mentioned in but I remembered I had to get a copy.

Reading the first edition of Code Complete became a series of wow moments. As a self taught programmer each chapter heralded a new revelation. I say proudly that this one book has had the biggest positive impact in my professional life as a programmer.

It is therefore very difficult to pick the best lesson I learnt from Steve McConnell. It finally came down to a choice of two; Valid Reasons to Create a Routine and the chapter on Debugging.

In the end I decided that it was Valid Reasons to Create a Routine. I won’t reveal the most important reason here but the answer will surprise you. It’s not to avoid duplicate code BTW.

If you want to know the answer I recommend you buy the book, I already know you will not regret it.

PL/SQL FizzBuzz

One of the popular exercises when learning a new language is to develop a FizzBuzz program.

This is where you print out the all the numbers from 1 to 100 except if a number is a multiple of 3 you print out Fizz and if the number is a multiple of 5 you print out Buzz. If a number is a multiple of both 3 and 5 then you print out FizzBuzz.

I came across this challenge for the first time recently as I learn C# so here is my version of FizzBuzz written in PL/SQL.

DECLARE

   fizz BOOLEAN := FALSE;
   buzz BOOLEAN := FALSE;

BEGIN

   FOR i IN 1 .. 100
   LOOP

      fizz := MOD(i, 3) = 0;
      buzz := MOD(i, 5) = 0;

      CASE

         WHEN fizz AND buzz THEN

            dbms_output.put_line('FizzBuzz');

         WHEN fizz THEN

            dbms_output.put_line('Fizz');

         WHEN buzz THEN

            dbms_output.put_line('Buzz');

         ELSE

            dbms_output.put_line(i);

      END CASE;

   END LOOP;
 
END;

 

Found a problem with the Oracle Documentation? Report it!

I was recently working through Chapter 9 of the 12c Database 2 Day Developer’s Guide and found the following bug in the sample code.

Here is the original code:

FUNCTION add_department
    ( p_department_name   IN departments.department_name%TYPE,
      p_manager_id        IN departments.manager_id%TYPE )
    RETURN departments.department_id%TYPE
  IS
    l_department_id departments.department_id%TYPE;
  BEGIN
    INSERT INTO departments ( department_id, department_name, manager_id )
    VALUES ( departments_sequence.NEXTVAL, p_department_name, p_manager_id );

    RETURN l_department_id;
  END add_department;

 
The function creates a new department and should return the newly created department id. Except it doesn’t. The variable used to return the newly created department id, l_department_id is never set so it will always returns null.

One solution to this bug is to use the Returning Into clause and the revised code would be:

...
    INSERT INTO departments ( department_id, department_name, manager_id )
    VALUES ( departments_sequence.NEXTVAL, p_department_name, p_manager_id )
    RETURNING department_id INTO l_department_id;
 
    RETURN l_department_id;
...

 
It’s a trivial issue so why I am writing about it? The reason is that seeing this problem I could have easily moved on and finished the chapter but I noticed the Feedback button and thought that others will also encounter this problem and may waste time wondering why the department id is not being returned.

OFeedback

After clicking the feedback button and using my OTN account (you also have the choice of remaining anonymous), describing the issue along with the suggested fix as described above I pressed submit and thought no more about it.

Within 24 hours I had an email reply from someone at Oracle (and someone not using a do_not_reply email address) thanking me for pointing out the error and assuring me it will be fixed in future editions. Whilst it remains to be seen if this will be done, the response to my feedback has left a very positive impression with me which made me glad I made the effort to report this issue.

So if you spot something within the masses of Oracle documentation, report it and help fix those broken windows.

The PL/SQL Continue statement

Introduced in Oracle Database 11.1 The PL/SQL CONTINUE statement allows you to skip the current  loop iteration. It can be used conditionally and unconditionally.

Unconditional Example

BEGIN

   FOR i IN 1 .. 10
   LOOP
 
      IF i = 2
      THEN
         CONTINUE; 
      END IF;
 
      DBMS_OUTPUT.PUT(i || ', ');
 
    END LOOP;

    DBMS_OUTPUT.NEW_LINE;

END;

In the example above, there is no output when the loop executes iteration 2, Running this code shows the following output.

1, 3, 4, 5, 6, 7, 8, 9, 10,

 Conditional Example

BEGIN

   FOR i IN 1 .. 10
   LOOP
 
      CONTINUE WHEN i = 2; 
 
      DBMS_OUTPUT.PUT(i || ', ');
 
   END LOOP;
 
   DBMS_OUTPUT.NEW_LINE;

END;

Again in this example there is no output for iteration 2. Running this code shows the following output.

1, 3, 4, 5, 6, 7, 8, 9, 10,

The conditional use removes the need for the IF statement used in the unconditional example and makes the code more concise without losing readability.

Summary

In this article I have shown with short code examples how to use the PL/SQL Continue statement.

This article has not discussed whether the use of a CONTINUE statement is a bad programming practise, there are already enough arguments discussions about that.

Source: The Continue Statement within the Database documentation