Howto write a perl package to interact with steemit

3년 전

steemit-perl.png

What Will I Learn?

  • How to create a simple modern perl project
  • Use Mojolicious to create simple object-oriented classes
  • Make web requests with to jsonrpc services
  • Interact with the steem api

Requirements

For this tutorial you will require:

Difficulty

Intermediate

Tutorial Contents

We will start with the basic setup and work out way up to interacting with the steemit api.

Basic project setup

First we create the basic folder structure we will require later on:

  • Bin
    • This will contain out executables
  • Lib
    • Here we will have the module which implements the functionality
  • Etc
    • Other required data will go in here

OK, so first we have to create out package. We will call it Steemit.pm and it goes into the lib folder of course

creating the library

vi lib/Steemit.pm
package Steemit;                                                                                                         
                                                                                                                         
use Modern::Perl '2017';                                                                                                 
use Mojo::Base -base;                                                                                                    
use Mojo::UserAgent;                                                                                                     
use Mojo::JSON qw(decode_json encode_json);                                                                              
                                                                                                                         
has url     => 'steemd.minnowsupportproject.org';                                                                                                             
has ua      =>  sub { Mojo::UserAgent->new };                                                                            
                                                                                                                                                                                                                                                  
1; 

We have here the initial package definition followed by the usage of some libraries. We use Modern::Perl since it will enable the latest perl feature set and language constructs. Further it will automatically use strictures and warnings which is just a sane thing to do.

As base for our project we are going to use Mojolicious Framework . This wills ave us a lot of time wringing object oriented code and has many helpers for accessing web resources and working with json based requests.

What have we donw so far?

  • We then defined our initial two attributes of our class.
  • A url where we will send our requests to
  • A user agent class that we will use to make the http requests

handling dependencies

We will use cpanminus for our perl based dependencies. It has a convenient way of using a cpanfile for installing everything we use. We will put it in the /etc folder of our project:

vi etc/cpanfile

requires 'Modern::Perl', undef;                                                                                          
requires 'Mojo::Base',   undef;                                                                                          
requires 'Test::More',   undef; 

We can denote here all the required modules and while it has more features that will warrant a own guide in the future we take the easy route and not “undef” as version and require all modules to have the latest version of them available.

We can then install the dependencies with

cpanm --v --installdeps ./etc/

adding a test

As good practice we will include a test file.

vi t/steemit.t
#!/usr/bin/env perl
use Modern::Perl '2017';
use Test::More;

use FindBin;
use lib "$FindBin::Bin/../lib";

use_ok('Steemit');
my $steem = Steemit->new;

isa_ok( $steem, 'Steemit', 'constructor will return a Steemit object');

done_testing;

We use Test::More to implement our tests. The use_ok statement will check if we can basically load the package. Then we create an instance of the class. Finally we check if the constructor of our cals indeed returns a valid object.

the client executable

We also want to use the library we are building. So we finally will create a script that will execute the functionalities.

vi bin/steemit_client.pl 
#/usr/bin/env perl
use Modern::Perl '2017';

use FindBin;
use lib "$FindBin::Bin/../lib";

use Steemit;

my $steem = Steemit->new;

say "Initialized Steemit client with url ".$steem->url;

It will again use our package, create a package and call our assessor to print out the url of the endpoint we are going to use.

This concludes our initial project setup. For reference everything can be found in this git branch But what has it to do with the steem api? Good question and we come to this now.

Building the Steemit integration

We first have to build a method to make our requests. For this we add the following to our lib/Steemit.pm

sub _request {                                                                                                           
   my( $self, $api, $method, @params ) = @_;                                                                             
   my $result = decode_json $self->ua->get( $self->url, json => {                                                        
      jsonrpc => '2.0',                                                                                                  
      method  => 'call',                                                                                                 
      params  => [$api,$method,[@params]],                                                                               
      id      => int rand 100,                                                                                           
   })->result->body;                                                                                                     
                                                                                                                         
   return $result->{result} if $result->{result};                                                                        
   if( my $error = $result->{error} ){                                                                                   
      die $error->{message};                                                                                             
   }                                                                                                                     
   #ok no error no result                                                                                                
   require Data::Dumper;                                                                                                 
   die "unexpected api result: ".Data::Dumper::Dumper( $result );                                                        
}                                                                                                                        

Now let's explain this a little bit. Steemit uses a jsonrpc api so we send json there and get json back. This is a simplistic view but covers the basics.

$self->ua->get will send a get request via the Mojo::UserAgent package. It already supports to send a json request body. We will always call the generic “call” method which needs the following parameters:

  • First an api section that we will call, Steemit supports here
    • Login_api
      • authentication
    • network_node_api
      • Methods about the network connectivity and state
    • network_broadcast_api
      • Sending messages ( voting, commenting……)
    • database_api
      • Read access to the steem blockchain
      • Within this tutorial we will focus only in the database_api since we can use it without hassling around with cryptography, security and so on.
  • Then the actual method we want to use. This specifies the actual functionality we want to use
  • An array of parameters that the method requires.

We use decode_json to unmarshal the response and translate json to perl data structures.

After this we handle our result. Either there is a “result” field defined, then we can return it. Otherwise we should have an “error” with a message attached. In case something is really off we will throw an exception with the whole result we have.

So now that we have the general method to make requests we will add a method to make one.

sub get_accounts {                                                                                                       
   my( $self, @params ) = @_;                                                                                            
   return $self->_request('database_api','get_accounts',@params);                                                        
}  

Then we can call it in our bin/steemit_client.pl . There can now just ad the simple lines:

use Data::Dumper;                                                                                                        
say Dumper( $steem->get_accounts(['utopian-io'])); 

This will return us a lot of information about the account of “utopian-io”:


$VAR1 = [
          {
            'can_vote' => bless( do{\(my $o = 1)}, 'JSON::PP::Boolean' ),
            'other_history' => [],
            'posting_rewards' => 3114052,
            'last_account_recovery' => '1970-01-01T00:00:00',
            'sbd_balance' => '0.000 SBD',
            'lifetime_market_bandwidth' => 0,
            'active_challenged' => bless( do{\(my $o = 0)}, 'JSON::PP::Boolean' ),
            'proxy' => '',
            'mined' => $VAR1->[0]{'active_challenged'},
            'last_account_update' => '2017-12-24T13:03:30',
            'savings_sbd_seconds' => '0',
            'average_market_bandwidth' => 0,
            'last_market_bandwidth_update' => '1970-01-01T00:00:00',
            'memo_key' => 'STM5uKByZ9z1QhbGG7AwjMuZzhTvSukJDhUcLhCZbwQkg1H46hUyd',
            'vesting_balance' => '0.000 STEEM',
            'savings_sbd_last_interest_payment' => '1970-01-01T00:00:00',
            'savings_sbd_seconds_last_update' => '1970-01-01T00:00:00',
            'curation_rewards' => 50833941,

This can already be used to access a lot of information on accounts. The example can be accesed via this git link

In a later example we will explore the steemit api further and learn how to expand this example to make something more useful out of it. So stay tuned .



Posted on Utopian.io - Rewarding Open Source Contributors

Authors get paid when people like you upvote their post.
If you enjoyed what you read here, create your account today and start earning FREE STEEM!
STEEMKR.COM IS SPONSORED BY
ADVERTISEMENT
Sort Order:  trending

20180219_120209.jpg

Hey @hoffmann I am @utopian-io. I have just upvoted you!

Achievements

  • You have less than 500 followers. Just gave you a gift to help you succeed!
  • This is your first accepted contribution here in Utopian. Welcome!

Suggestions

  • Contribute more often to get higher and higher rewards. I wish to see you often!
  • Work on your followers to increase the votes/rewards. I follow what humans do and my vote is mainly based on that. Good luck!

Get Noticed!

  • Did you know project owners can manually vote with their own voting power or by voting power delegated to their projects? Ask the project owner to review your contributions!

Community-Driven Witness!

I am the first and only Steem Community-Driven Witness. Participate on Discord. Lets GROW TOGETHER!

mooncryption-utopian-witness-gif

Up-vote this comment to grow my power and help Open Source contributions like this one. Want to chat? Join me on Discord https://discord.gg/Pc8HG9x

Thank you for the contribution. It has been approved.

You can contact us on Discord.
[utopian-moderator]

·

thank you so mutch :D
i guess the third time is the charm

This is great! I'm working with Python and steempy but I'd much rather be working with Perl.

Is there a cpan version of this, by any chance? If not please accept my vote for one (I'd upvote this post, but it was 9 days ago). In fact I could help with it, if you're interested.

·

hi,

in fact i want to make it a cpan module in the future but first i want to line out the details on how it works.
i guess in about 4-5 posts from now it should become a cpan module ;)

·
·

Excellent! Looking forward to it, just followed you.

·

Hmm, the "use Modern::Perl '2017';" line is failing with this error:

Feature bundle "5.24" is not supported by Perl 5.22.1 at /home/steemitdev/perl5/lib/perl5/Modern/Perl.pm line 38.
BEGIN failed--compilation aborted at bin/steemit_client.pl line 2.

I'm using an Ubuntu 16.04.3 (LTS) VM. Just did "sudo apt update" and "sudo apt upgrade" but the newest version of Perl available is 5.22. Don't want to upgrade to a newer Ubuntu version that's not LTS (long-term support).

Aha! I see that your link to the Perlbrew tutorial should resolve my issue. Great, thanks!

·
·

Hmm, guess not. Got everything installed, but now when I try running the example client script (bin/steemit_client.pl), I get this error:

Initialized Steemit client with url steemd.minnowsupportproject.org
Malformed JSON: Expected string, array, object, number, boolean or null at line 0, offset 0 at /home/steemitdev/projects/example.com/steemit/Steemit/bin/../lib/Steemit.pm line 13.

Below is my entire lib/Steemit.pm. Note that line 13 is the second line in the sub _request -- it's the "my $result = decode_json [...]" line.

package Steemit;                                                                                                         
                                                                                                                         
use Modern::Perl '2017';                                                                                                 
use Mojo::Base -base;                                                                                                    
use Mojo::UserAgent;                                                                                                     
use Mojo::JSON qw(decode_json encode_json);                                                                              
                                                                                                                         
has url     => 'steemd.minnowsupportproject.org';                                                                                                             
has ua      =>  sub { Mojo::UserAgent->new };                                                                            
                                                                                                                                                                                                                                                  
sub _request {                                                                                                           
   my( $self, $api, $method, @params ) = @_;                                                                             
   my $result = decode_json $self->ua->get( $self->url, json => {                                                        
      jsonrpc => '2.0',                                                                                                  
      method  => 'call',                                                                                                 
      params  => [$api,$method,[@params]],                                                                               
      id      => int rand 100,                                                                                           
   })->result->body;                                                                                                     
                                                                                                                         
   return $result->{result} if $result->{result};                                                                        
   if( my $error = $result->{error} ){                                                                                   
      die $error->{message};                                                                                             
   }                                                                                                                     
   #ok no error no result                                                                                                
   require Data::Dumper;                                                                                                 
   die "unexpected api result: ".Data::Dumper::Dumper( $result );                                                        
}    

sub get_accounts {                                                                                                       
   my( $self, @params ) = @_;                                                                                            
   return $self->_request('database_api','get_accounts',@params);                                                        
} 

1;

I'm unclear on why it's failing. I'll try "taking apart" the code, so it's doing one thing at a time and see if I can find it.

Okay, I've updated the module and will reproduce the new version below as well, and the new output -- which is telling me that perhaps the URL needs to be changed?

[Oops! While hitting "`" three times, then Enter, I typoed and hit Tab for one of them -- and then the "Enter" posted this! So, I'll continue it in a child comment.]

·
·
·

The new module:

package Steemit;                                                                                                         
                                                                                                                         
use Modern::Perl '2017';                                                                                                 
use Mojo::Base -base;                                                                                                    
use Mojo::UserAgent;                                                                                                     
use Mojo::JSON qw(decode_json encode_json);                                                                              
                                                                                                                         
has url     => 'steemd.minnowsupportproject.org';                                                                                                             
has ua      =>  sub { Mojo::UserAgent->new };                                                                            
                                                                                                                                                                                                                                                  
sub _request {                                                                                                           
   my( $self, $api, $method, @params ) = @_;                                                                             
   my $get_result = $self->ua->get( $self->url, json => {                                                        
      jsonrpc => '2.0',                                                                                                  
      method  => 'call',                                                                                                 
      params  => [$api,$method,[@params]],                                                                               
      id      => int rand 100,                                                                                           
   })->result->body;
   print "INFO: result from get is [$get_result]\n";
   my $result = decode_json $get_result;
                                                                                                                         
   return $result->{result} if $result->{result};                                                                        
   if( my $error = $result->{error} ){                                                                                   
      die $error->{message};                                                                                             
   }                                                                                                                     
   #ok no error no result                                                                                                
   require Data::Dumper;                                                                                                 
   die "unexpected api result: ".Data::Dumper::Dumper( $result );                                                        
}    

sub get_accounts {                                                                                                       
   my( $self, @params ) = @_;                                                                                            
   return $self->_request('database_api','get_accounts',@params);                                                        
} 

1;

First, I added the "my $get_result" line, and moved the "my $result = decode_json $get_result;" to below it. This failed in the same manner, so then I added the "print" statement in between them, which gave me:

INFO: result from get is [<html>
<head><title>502 Bad Gateway</title></head>
<body bgcolor="white">
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.10.3 (Ubuntu)</center>
</body>
</html>
]

Note that one benefit of surrounding the variable I'm outputting with square brackets, is that I see there's a trailing CR after "". That doesn't really affect this result, but it has given me insight into what's happening, in other instances.

Anyway, it's telling me there's a bad gateway. So, let's try loading that URL in a browser, and see what happens. Okay, same result, just in web format. That URL was "steemd.minnowsupportproject.org", and I vaguely know about some API changes breaking usable work. So I tried "api.minnowsupportproject.org" but that wasn't found.

So then I went to "minnowsupportproject.org" -- and see that their home page has a bunch of broken functionality on it as well!

So I guess I'm blocked now. I'll finish tabulating the votes manually, I guess. Thanks though, this has been educational! :)

·
·
·
·

hi,

fist of all thanks for giving it a try.
It seems that https is a thing now so we need some more dependencies.
i have pushed an update into the linked git repositories.

we need as dependancies

sudo apt-get install libssl1.0-dev zlib1g-dev

then we need to install ssl support for perl

cpanm IO::Socket::SSL

the issue with the modern perl is that 2017 is only available in perl5.26 with earlyer versions you can just use it without a date.
Alternatively you can check out my earlyer guide on how to install your local independant perl installation in your home directory.

I have then switched ( also in git ) the url of the steemd service to

https://rpc.steemliberator.com

wich seemed to work find.
But if you want more stable results you may want to think about running you local steemd node.

I had my nodes from this article

best regards
hoffmann

·
·
·
·
·

Okay, now I can upvote you. :) Here are three, thank you so much for this project. I determined the IO::Socket::SSL dependency myself, and that other thing I needed to install (looks like you forgot to add the package name in the above? I have forgotten it, and it's not in ~/.bash_history, and I don't want to exit all my "screen" windows right now to find it... Perhaps there's a way to ask dpkg what the last package installed was? Anyway...).

I just tested with https://rpc.steemliberator.com/ and it worked! That's so awesome, thanks again!

Although I had to tabulate my results manually in the contest I ran that just ended, this is great because I will be able to use it in the future.

·
·
·
·
·
·

hi,

thanks a lot and sorry, i forgot to paste the packages.
i have updated the post.

·
·
·
·
·
·

hi,

i found a good site https://www.steem.center/index.php?title=Public_Websocket_Servers wich seems to be quiet up to date