I cannot get right Japanese stock price data from MSN.
My online quotes setting is as a following.

URL: http://ca.moneycentral.msn.com/investor ... ?symbol=JP:%1
Symbol: (blank)
Price: (\d+,\d+) [+-]\d+ [+-]\d+.\d+\%
Date: Time of last trade (\d+/\d+/\d+)
Date Format: %d %m %y

KMyMoney fetch wrong price data as below.

i.e. TOYOTA: trading symbol = 7203

Fetching URL http://ca.moneycentral.msn.com/investor ... =JP:7203...
Symbol found:
Price found: 3.340 (3.34)
Date found: 金 4 8 2011
Price for 7203 updated (id E000003)

But TOYOTA's right price is 3340 at 10 April.

How to get right price?

Thanks

KMM is interpreting the comma as a separator of decimals rather than thousands. I could be wrong here, but I think KMM follows your system settings (Common Appearance -> Locale -> Numbers, depending on your OS).

Also see this topic.

I don't think that this will help. From the source:

Code: Select all

      // Deal with european quotes that come back as X.XXX,XX or XX,XXX
      //
      // We will make the assumption that ALL prices have a decimal separator.
      // So "1,000" always means 1.0, not 1000.0.


Well, this assumption is not valid for the Japanese quotes. Guess we have to take care of them in the future.

This is a Canadian website and a Japanese quote. So, is that comment still valid?

Shouldn't toki use a Japanese site, such that his local settings match the format of the web site?


BTW I can't get a quote using toki's URL.

You need to activate the 'Skip HTML' option. Any site not using a thousand separator should work.

Thanks for your reply, zebulon and ipwizard.

I changed decimal and thousand separator in OS local settings, but still got wrong price. I doubt fetch data is using comma as a thousand separator.

Why I used Canadian site for fetching. When I used Japanese site, I cannot recognize there are many matter or differences, multi byte character code, format, regular description. Then I used Canadian MSN that can show us Japanese stock price.

Valid URL(This time I used URL function in this forum)
http://ca.moneycentral.msn.com/investor/quotes/quotes.aspx?symbol=JP:7203

URL setting in Online quotes

Code: Select all

http://ca.moneycentral.msn.com/investor/quotes/quotes.aspx?symbol=JP:%1

By the way, if you know other stock price site that can show Japanese stock price in Europa, let me know it.

Thanks

If I may ask, what is the trading symbol of this stock, and how did you define the Online Update (investment detail wizard)? I'd like to try a couple of things.

zebulon wrote:I'd like to try a couple of things.

You can certainly try, but you won't succeed unless you are able to remove the optional comma. The code that is executed also does this

Code: Select all

      // Remove all non-digits from the price string except the last one, and
      // set the last one to a period.


before the string is converted into a numeric value.

zebulon, Toyota's trading symbol is 7203.

By the way, I will be looking for other stock site that is not using a thousand separator.

ipwizard wrote:You can certainly try, but you won't succeed unless you are able to remove the optional comma.

My intention was to try using Finance::Quote. The scripts it uses are easy to modify if necessary. However, I did not yet succeed getting a quote at all this way. Concerning MSC.CA, I can't get an online quote either. Does anybody have an idea what I am doing wrong? I defined The Toyota investment with trading symbol "JP:7203", full name "TOYOTA MOTOR CORP". In he online update panel, I do not check Finance::Quote but select MSN.CA. It is unable to get a price though.

BTW: the URL echoed in the status window is correct and works if pasted in the address bar of a browser.

It seems that Tokyo stock exchange commonly uses commas as thousand separator: I defined a new online quote source for Bloomberg, same issue.

@ipwizard: I second the suggestion to address the 'comma-is-thousands-separator' issue as soon as possible.

@zebulon: I also like to try Finance::Quote as workaround. It seems that these scripts are highly adaptable to any kind of format. What did you try already?

Al_ wrote:@zebulon: I also like to try Finance::Quote as workaround. It seems that these scripts are highly adaptable to any kind of format. What did you try already?

I mainly use it for the French stock market. The problem is that the scripts need to be adapted every time a web site changes its layout. You can find updated scripts on the web, e.g. on github. I use a perl script to test these Finance::Quote scripts

Code: Select all

#!/usr/bin/perl
use Finance::Quote;
my $q = Finance::Quote->new();
$stock = 'IBM';
my %data = $q->fetch('Yahoo',$stock);
print "I found for ".$stock.": ".$data{$stock, 'last'}. "\n";
print "error flag: " .$data{$stock, 'errormsg'}. "\n";
print "success flag: " .$data{$stock, 'success'}. "\n";

Thanks. In addition to the yahoo module, that you use in your example, I also have

AEX.pm
DWS.pm
LeRevenu.pm
Troweprice.pm
AIAHK.pm
Fidelity.pm
ManInvestments.pm
Trustnet.pm
ASEGR.pm
FinanceCanada.pm
Morningstar.pm
TSP.pm
ASX.pm
Finanzpartner.pm
NZX.pm
TSX.pm
BMONesbittBurns.pm
Fool.pm
Platinum.pm
Union.pm
Bourso.pm
FTPortfolios.pm
SEB.pm
UserAgent.pm
Cdnfundlibrary.pm
GoldMoney.pm
StockHouseCanada.pm
USFedBonds.pm
Cominvest.pm
HEX.pm
Tdefunds.pm
VWD.pm
Currencies.pm
HU.pm
Tdwaterhouse.pm
Deka.pm
IndiaMutual.pm
Tiaacref.pm
ZA.pm

It seems I need to write an additional module for Japanese stocks, e.g. MSN-CA-for-Japan.pm or Tokyo-Stock-Exchange.pm
Not what i hoped for, but a reason to start perl (in addition to C++ and Basic dialects). Or has anybody already a module for MSN.CA ?

Meanwhile I wrote a perl script, i.e., a Finance:Quote module, to retrieve stock prices from the official website of the Tokyo stock exchange (http://www.tse.or.jp/english). This website has two shortcomings:

• I nowhere find the date of the last trade, so my script assumes 'today' (i.e., until a newer transaction is recorded, it uses the last price; 'last observation carried forward' principle); or do you see the trade date on, e.g., http://quote.tse.or.jp/tse/quote.cgi?F=listing%2FEDetail1&MKTN=T&QCODE=6758 ?
• The site is sometimes not reachable, possibly times out. The script tries twice, with slightly different http requests (both should be equal, but for vodoo reasons, sometimes one seems to work better than the other)

If you find the script useful, please let me know (by posting in this forum or by PM). Equally or more important, let me know if you find bugs, or if you know a more reliable webpage for Tokyo stock exchange prices.

File: save the code below in TokyoStockExchange.pm, then proceed as per instructions in that file (see bottom half).

Code: Select all

#!/usr/bin/perl -w
#
#    Copyright (C) 2011, Al_
#
#    This program is free software; you can redistribute it and/or modify
#    it under the terms of the GNU General Public License as published by
#    the Free Software Foundation; either version 3 of the License, or
#    (at your option) any later version.
#
#    This program is distributed in the hope that it will be useful,
#    but WITHOUT ANY WARRANTY; without even the implied warranty of
#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#    GNU General Public License for more details.
#
#    You should have received a copy of the GNU General Public License
#    along with this program; if not, write to the Free Software
#    Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA
#    02111-1307, USA
#
# Purpose: to retrieve stock prices from Tokyo Stock Exchange website
# To be used in conjunction with Finance::Quote


use strict;
package Finance::Quote::TokyoStockExchange;
#use Finance::Quote;
#use warnings;
#use strict;

sub methods { return ( tse => \&tokyostockexchange,
                       tokyostockexchange => \&tokyostockexchange ); }

{
  my @labels = qw/name, last, date, currency, exchange, success, errormsg, source/;
  sub labels { return (tse => \@labels,
                       tokyostockexchange => \@labels); }
}

sub tokyostockexchange {
   my $quoter = shift; # The Finance::Quote object
   my @stocks = @_;
   my %info;
   my $tse_base_url1 = "http://quote.tse.or.jp/tse/quote.cgi?F=listing%2FEDetail1&MKTN=T&QCODE=";
   my $tse_base_url2 = "http://quote.tse.or.jp/tse/quote.cgi?F=listing/EDetail1&MKTN=T&QCODE=";
   my $ua = $quoter->user_agent;
   my($day, $month, $year)=(localtime)[3,4,5];
   foreach my $stock (@stocks) {
      my $tse_full_url = $tse_base_url1 . $stock;
      $info{$stock,"currency"} = "JPY";   # all in Japanese Yen
      $info{$stock,"exchange"} = "Tokyo Stock Exchange";
      my $response = $ua->get($tse_full_url);
      my $http_success = $response->is_success;
      $http_success = $http_success + 0; # force numified
      if ($http_success == 0) {
         $tse_full_url = $tse_base_url2 . $stock;
         $response = $ua->get($tse_full_url);
         $http_success = $response->is_success;
         $http_success = $http_success + 0; } # force numified
      $info{$stock,"source"} = "Finance::Quote::TokyoStockExchange, data from " . $tse_full_url;
      my $webpage = $response->decoded_content;
      $webpage = $webpage . "";      # force stringified
      $info{$stock,"success"} = 0;   # overridden in case all steps were successful
      if ($http_success == 0) { $info{$stock,"errormsg"} = "HTTP error for " . $tse_full_url . ": " . $response->message . "; status line: " . $response->status_line; }
      elsif ($webpage eq "") { $info{$stock,"errormsg"} =  "webpage " . $tse_full_url . " is empty"; }
      else {                        # parse webpage
         $webpage=~m{<td bgcolor="#6A6A6A" width="510"><font color="#FFFFFF">([a-zA-Z\.,;:-_ ]*)</font></td>};
         $info{$stock,"name"} = $1;
         $webpage=~m{<td align="middle" width="24%" bgcolor="#FFDDB7"><b>Recent Price<br>\(Trade (Time|Date)\)</b></td>\s*<td align="right" width="26%"><b>(\d*,\d*)\s};
         if ($1 eq "Time") { $info{$stock,"date"} = sprintf("%02d",$month+1) . "/" . sprintf("%02d",$day) . "/" . sprintf("%02d",$year-100); } # today
#         elsif ($1 eq "Date") { $info{$stock,"date"} = "retrieve date from webpage"; } # MM/DD/YY format, does not seem to occur ever
         else { $info{$stock,"date"} = ""; $info{$stock,"errormsg"} = "unexpected format of webpage, data incomplete or missing"; }
         my $lastprice = $2;
         $lastprice = $lastprice . "";      # force stringified (could be numeric if no ',')
         $lastprice =~ s{,}{}g;
         $info{$stock,"last"} = $lastprice;
         $info{$stock,"success"} = 1; }}
   return wantarray() ? %info : \%info; }


=head1 NAME

Finance::Quote::TokyoStockExchange - Obtain quotes from Tokyo Stock Exchange.

=head1 SYNOPSIS

    use Finance::Quote;

    $q = Finance::Quote->new;

    %info = Finance::Quote->fetch("tokyostockexchange","6758");  # Only query TokyoStockExchange
    %info = Finance::Quote->fetch("tse","6758"); # Failover to other sources OK.

=head1 DESCRIPTION

This module fetches information from the "Tokyo Stock Exchange" at http://quote.tse.or.jp.

This module is loaded by default on a Finance::Quote object if the environment variables are set accordingly.
It's also possible to load it explicity by placing "tse" in the argument list to Finance::Quote->new().

This module provides both the "tokyostockexchange" and "tse" fetch methods.
   "tse" has failover with future sources for Tokyo stocks.
   "tokyostockexchange" fetches information only from the Tokyo Stock Exchange website.

Use instructions:
   Place this file in folder */Finance/Quote (full path on my machine: /usr/share/perl5/Finance/Quote)
   Search http://www.tse.or.jp/english/ for correct symbol
        typically 4 digit number, termed 'Code';
        only diplayed once clicked "Display of stock price", NOT the 5-digit number on the initial search results page
   Use this code as lookup symbol in your preferred, Finance::Quote enabled application
   If using kmymoney, that application needs to instruct Quote::Finance to load Quote::Finance::TokyoStockExchange,
        this is done e.g., by starting kmymoney using the following command line:
        env FQ_LOAD_QUOTELET="-defaults TokyoStockExchange" kmymoney

Information obtained by this module may be covered by Tokyo Stock Exchange
terms and conditions See their website for details.

ISSUES

The website of Tokyo Stock Exchange is instable and often returns "Bad Gateway". A web browser such as Firefox
   then displays "The specified CGI application misbehaved by not returning a complete set of HTTP headers".
   If this issue occurs, then it persists for some period of time (minutes?).
   Please send complaints to Tokyo Stock exchange, not to me ...

I was unable to locate date of last trade (only time!?!), thus, this module always assumes 'today'.

=head1 LABELS RETURNED

The following labels may be returned by Finance::Quote::TokyoStockExchange :
name, last, date, currency, exchange, success, errormsg, source.

=head1 SEE ALSO

Tokyo Stock Exchange (English web site), http://www.tse.or.jp/english

=cut