Push your package to pip

Finally after a long time .. a PR on my git project  https://github.com/Neetuj/python-wns
reminded me that i still need to push it to pip.

Given the hits I get on my notification entry (most popular so far) push-notifications-on-apns-gcm-WNS 
I felt bad for not putting in the effort before to push python-wns to pip
Anyways I finally did it today and now you all can avail the power of pip for python-wns by

pip install python-wns

Now if you are the one who are not interested in my pyton-wns package but want to push your own package here are the steps I followed  (order can be important so follow it :)

• Register on https://testpypi.python.org/pypi
• Register on https://pypi.python.org/pypi
• Confirm all verification emails for the new accounts 
• create a .pypirc file "vim ~/.pypirc" which looks like this 
• [distutils]
  index-servers =
    pypi
    test
  
  [pypi]
  repository=https://pypi.python.org/pypi
  username=name
  password=pwd
  
  [test]
  repository=https://testpypi.python.org/pypi
  username=name
  password=pwd
• package your library like this  

• python-wns/
    setup.py
    setup.cfg
    LICENSE.txt
    README.md
    wns/
      __init__.py

      wnslib.py 

   
•  The setup.py needs to be in a certain format you can checkout mine here setup.py
• Then test 
•  python setup.py register -r test
•  proceed if you get Server response (200): OK
• python setup.py sdist upload -r test
• proceed if you get Server response (200): OK
• python setup.py register sdist upload
• and you are done!!  test "pip install python-wns"
  

Install docker on MAC

I had docker on my ubuntu and I do not remember going through much for installing docker on ubuntu .

But for MAC i had to follow these instructions ( in case anybody is looking for these)

    brew install docker
    brew install boot2docker
    boot2docker up
    boot2docker init
    boot2docker up
    export DOCKER_HOST=tcp://192.168.59.103:2375
    export DOCKER_CERT_PATH=/Users/yours/.boot2docker/certs/boot2docker-vm
    export DOCKER_TLS_VERIFY=1
    boot2docker ip

Debugging openstack ....logs , services e.t.c

Something is broken ...
lets say heat is not starting up ..
what do we do ?

calm down!
check logs ( /var/log/heat-api.log,  /var/log/heat-api-cfn.log, /var/log/heat-engine.log)

Do you see any errors there ?? ( most of the times this should be sufficient)

but what if there is not error in the logs.. since the service is not even starting ?
try restarting the service(systemctl comes to rescue)

systemctl list-units |grep heat

systemctl status openstack-heat-api.service

systemctl status openstack-heat-api-cfn.service

systemctl status openstack-heat-engine.service

Just getting a failed status but no details ?try debugging

journalctl -u openstack-heat-api-cfn.service

here are few links which will help

https://access.redhat.com/articles/systemd-cheat-sheet

https://cdn.rawgit.com/ruzickap/linux.xvx.cz/gh-pages/files/systemd_cheatsheet/systemd_cheatsheet.pdf

Beginners guide to cloud and openstack

Disclaimer :-- all the content is sourced from various pages on openstack .. I am just putting together that information in a crash course :)

 

What is Cloud?

Let's quickly review just what exactly we mean by cloud technologies?
 Cloud technologies are built on existing technologies such as virtualization and clustering to virtualize hardware, software, storage, and networking resources into flexible units that are quickly allocated to meet demand. So rather than the old static model of dedicated hardware servers for various tasks, and static network and storage configurations, all of those formerly specialized devices are assimilated into a common resource pool. It's a more efficient use of hardware, and very fast to scale up or down according to demand. You can even configure self-service for users so they can grab whatever they need when they need it.
Types of cloud ?

• Private
• Public
• Hybrid
• 
  
 

 

All computing resources are shareable in a cloud, and there are three basic service models:

• SaaS, software as a service
• PaaS, platform as a service
• IaaS, infrastructure as a service

What is Openstack? 
Open Source Cloud Computing platform (short answer).
OpenStack is a cloud operating system that controls large pools of compute, storage, and networking resources throughout a datacenter, all managed through a dashboard that gives administrators control while empowering their users to provision resources through a web interface.
https://en.wikipedia.org/wiki/OpenStack (long answer)

What are  Openstack components? 

core projects (short answer):

• Identity (Keystone)
• Dashboard (Horizon)
• Orchestration (Heat)
• Metering (Celiometer)
• Object stroage (Swift)
• Image service (Glance)
• Networking (Quantum now Neutron)
• Compute (Nova)

https://en.wikipedia.org/wiki/OpenStack (not so long answer)
https://www.openstack.org/software/ (long answer)

How do these Openstack Components talk to each other?

1. Messaging Queues: AMQP is the messaging technology chosen by the OpenStack cloud.The OpenStack components such as Nova , Cinder , Quantum communicates internally via AMQP(Advanced Message Queue Protocol) and through eachother using REST-call. The AMQP broker, RabbitMQ , sits between any two internal Openstack components and allows them to communicate in a loosely coupled fashion i.e its components has, or makes use of, little or no knowledge of the definitions of other separate components. More precisely, Nova components (nova-api, nova-scheduler , nova-compute) use Remote Procedure Calls (RPC) to communicate to one another.
   In OpenStack the Nova ,Cinder and Quantum implements RPC (both request+response, and one-way, respectively nicknamed ‘rpc.call’ and ‘rpc.cast’) over AMQP by providing an adapter class which take cares of marshaling and unmarshaling of messages into function calls. Each Nova components (for example api ,compute, Scheduler, etc.) , Cinder components ( for example volume, Scheduler) , Quantum Components( for example quantum-server , agents ,plugins )  create two queues at the initialization time, one which accepts messages with routing keys ‘NODE-TYPE.NODE-ID’ (for example compute.hostname) and another, which accepts messages with routing keys as generic ‘NODE-TYPE’ (for example compute) .
2.  

Beginner's links to find out about openstack

https://opensource.com/business/14/2/openstack-beginners-guide

https://eloiguillaud.wordpress.com/2014/10/12/cloud-computing-solutions-comparative-study-openstack-13/

https://www.linux.com/learn/docs/720624-introduction-to-openstack-from-zero-to-domination

https://www.linux.com/learn/tutorials/866101-how-to-test-drive-openstack

https://wiki.openstack.org/wiki/Main_Page

playing with openstack HEAT : beginners introduction

• Install heat
• Heat templates reside here

1.  https://github.com/openstack/heat-templates
2. which can be seen on the vm its in /vagrant/heat_templates
3. there are test templates in root dir as well ex:- /root/heat_test_template.yml

• on the controller vm this is how to use heat to spin up nodes
  
  1. heat stack-create -f heat_templates/test.yml test
  2. once its created you should be able to see it using CLI like this :-
     [root@controller-01 vagrant(keystone_admin)]$ heat stack-list
     
      +--------------------------------------+------------+---------------+----------------------+
     | id                                   | stack_name | stack_status  | creation_time        |
     +--------------------------------------+------------+---------------+----------------------+
     | xxxx-xxxxx | test       | CREATE_FAILED | 2015-11-16T20:59:49Z |
     +--------------------------------------+------------+---------------+----------------------+
  3. on horizon you can see it here 
  
  

          7.  If you click the "stack name" link you can get into more details of your stack.
          8. There are Actions like "resume stack, suspect stack , delete stack" e.t.c that you can perform
         9.  You can get different details on the stack on horizonw including the template itself and you can change the template
     

• General Heat Resources :-

http://docs.openstack.org/developer/heat/template_guide/

https://wiki.openstack.org/wiki/Heat

https://github.com/openstack/heat

why use HSM(Hardware Security Module) ? What to look for in a HSM?

Let's answer first why ? Why a HSM should be used?
We all know security has sort of become a myth. Specially in today's interconnected, big data, cloud crazy world. The questions is How to secure data ? documents ? keys? passwords?

Very sensitive applications have to protect several different forms of data and in some industries (banking, health, complaint heavy ones) we absolutely can not overlook the possibility of security compromises.

Solutions with various encryption technologies came up when the field of security was burgeoning but they were only safe as long as the keys are safe . How do you keep the keys safe? Store your software key in some software? NAAA!! that won't work.
Store them in tapes like Millitary ? NAAA!! that won't work
Store them in a tamper proof hardware with multiple level of authentications? May be !!  atleast to some extent!

That's where Hardware Security Module HSM comes into play (added bonus it can help you clear  your industry's security compliance needs )

Now that we know why ( and there are more use cases here  if you still need more reasons) the nest question is what to look for in an HSM?

Lets list the technical factors that might help  :-

• Performance -  encryption/decryption/key generation/signing, symmetric, asymmetric, decide based on your application needs
• Redundancy - consider the failover cases, how often can it happen? How easy is it to replace it? Assess the impact of it on your operations and have a plan in place and in case that happens or look for a more robust one. Dual NIC or not , down time e.t.c are questions to be asked.
• Backups - No matter how robust HSM is you can not afford to not think about backups because of the very real possibility of loosing access to your data forever. Consider how easy it is to backup your data (in a secure way ofcourse) , restore your data in HSM. 
• Scalability -  Most HSMs have limit on slots available, number of keys you can stores, number of users you can add e.t.c e.t.c Have the scale in mind. What if you need to scale up or down ( for cost cutting) 
• Procedures - Lets face it .. using HSM is not gonna make things simple .. its gonna make them more complex.  The real cost of using HSM is felt in operations because of the complexity that is introduced.A good HSM will come with a detailed usage manual which describes how things should be done to make sure its used correclty. It's not the hardware which matters, but how you use it. So consider APIs, Device management options , customer support , policy level controls , configuration options  e.t.c
• Physical/Logical Security -  Ofcourse all HSMs should be able to tackle physical or logical hacking in to the hardware/software of HSM. Most HSMs as a result have tamper proof cases, hardcase ,  multiple level of authentications and "wipe everyhting" attitude for intrusion attacks. 
• Complaince - If you have any compliance requirements make sure HSM supports it.