PHP Application to Estimate the Net Value of SZ162411
Aug 18, 2015
As my CSR stock is turning into cash soon, I am considering to use the USD to trade XOP while using my other CNY to trade SZ162411 together.
I was watching stock quotations on Yahoo and Sina everyday, and always need to click on the calculator application to convert the price between XOP and SZ162411, soon I got bored.
Later I thought of my first PHP application 5 years ago, and decided to write my second PHP application.
It was planned to put all the usual stock quotations together, and to estimate SZ162411 net value based on ETF-SPDR S&P (AMEX:XOP), ^SPSIOP and USD/CNY exchange rate.
Today the first version is released, and I am writing this record of programming details for future reference.
Using Yahoo Finance API for US stocks (15 minutes delay): http://finance.yahoo.com/d/quotes.csv?s=XOP+%5ESPSIOP&f=l1t1p2nd1p
Using Sina realtime source for Chinese stocks, oil futures and forex: http://hq.sinajs.cn/list=sz162411,hf_CL,USDCNY
At first I found that fopen will always fail to open those urls, maybe my Yahoo Web Hosting does not support allow_url_fopen.
I searched and found info at b1g dot de provided a HTTPRequest class to solve this problem, copied to file /php/class/http_req.php.
Functions using HTTPRequest to read stock quotes, display green for gain and red for loss are located in /php/stock.php.
Other php code only for this tool are located in /woody/res/php/_lof.php. The name directly comes from the type of SZ162411, which is called LOF, a Chinese special ETF like fund.
To optimize web response time, I used 2 files xop.txt and sz162411.txt to save last updated Yahoo and Sina stock data.
The following are checked:
- New request in the same minute using data in original files directly.
- Using xop.txt for US stock data after US market closed.
- Using sz162411.txt for Chinese stock data after Chinese market closed.
Similarly, oil future data is stored in file cl.txt
, and USDCNY forex data in usdcny.txt
Aug 20, 2015
To set up my own trading rules, and to avoid following the market all night, I plan to trade XOP simply on SMA values.
This version added the premium of current trading price comparing with XOP estimation of SZ162411 net value,
and the days in past 100 trading days did the estimated SMA value fitted in the trading range.
Using Yahoo stock historic data:
The code is in /php/stock/stockhistory.php.
The historic data of XOP only need to update once a day, but I do not konw when will Yahoo update the current day data, so I update it once an hour stupidly.
Using file xop_500.txt to store it.
The official fund net value only need to update once a day too.
Using file f_162411.txt for official SZ162411 net value from http://hq.sinajs.cn/list=f_162411.
The code is in /php/stock/fundref.php.
Using debug file debug.txt for temp data checking.
Aug 21, 2015
After released the link of this tool, I checked the data of Google Analytics yesterday. During the 3 days there are 584 visits from 289 different IP address.
Unlike the usual Palmmicro visitors, 1/3 of the tool visitors used mobile phone. So I added the UI optimization for mobile phone in a hurry.
Using Mobile-Detect to detect if it is a mobile phone visit.
Following the developer's advice, I copied the code github and put it separately in /php/class/Mobile_Detect.php.
Aug 27, 2015
The best way to organize the source code is to finish the English version and to write a few more similar LOF net value estimation software.
I bought 2 HK ETF related LOF with recent stock market crash,
and added CHINA ASSET HANG SENG and E FUND H-Share net value tool.
Although I only watched US market crash, I still added BOSERA S&P 500 net value tool for possible future usage.
The original one file /woody/res/php/_lof.php now becomes 5 files:
||Put html unrelated php code together
||SMA related functions
||/woody/res/sz162411.php, sz162411cn.php, sz159920.php, sz159920cn.php etc
||LOF related functions
||Stock symbol ralated data and code
||Code to generate page links
Sep 13, 2015
Integrated with my first PHP application. After login, user can now input related stock transaction record.
And we make SZ162411 and XOP arbitrage analysis based on those record.
The input and handling of stock transaction record is in file /woody/res/php/_edittransactionform.php and _submittransaction.php.
Woody's Web Tool is modified to generate _submitXXX.php file automatically when insert copy of a _editXXXform.php file.
Nov 7, 2015
Continue to organize code, add similar LOF net value tool for GUOTAI NASDAQ-100, NASDAQ-100,
Southern Hong Kong, JIASHI H-Share, GUOTAI COMMODITY, BRIC,
NUOAN Energy, HUAAN S&P GLOBAL OIL and ICBC Natural Resources.
Steps to add new LOF tools:
- Add the new LOF code in function LofGetSymbolArray, for tools navigation link.
- Add LOF related ETF symbol and index symbol in function LofGetEtfSymbol and LofGetIndexSymbol.
price comparing tool for Aluminum Corporation of China
, China Unicom
, China Life Insurance
, Sinopec Shanghai Petrochemical
and Yanzhou Coal Mining
file is added for ADR. The commone stock data part of ADR and LOF is moved to StockReference
class in file /php/stock/stockref.php
used in _LofGroup
class in file php/_lof.php
class in file php/_adr.php
Nov 24, 2015
Continue to use web tools to replace calculator works, add price calculation tool for Future ETF like USO/UWTI/DWTI based on NYMEX CL.
Including similar from NYMEX NG to UNG/UGAZ/DGAZ price, from COMEX GC to GLD/DGP/DZZ price,
from COMEX SI to SLV/AGQ/ZSL price.
Same way as php/_adr.php and php/_lof.php, add _FutureGroup class and php/_future.php file.
Sina Realtime US Stock Data
Dec 13, 2015
With the help of abkoooo, now using Sina realtime US stock data (http://hq.sinajs.cn/list=gb_xop) to replace original Yahoo data which has 15 minutes delay.
Now XOP data in file xop.txt. ^SYSIOP data still uses Yahoo and in a separated file _spsiop.txt.
The code of StockReference class in file /php/stock/stockref.php is a mess now, I doubt if I can still understand it in the future.
Jan 8, 2016
SZ162411 history net value table was added following the advice from airwolf2026.
The most recent a few records are displayed in the current page, add files netvaluehistory.php, netvaluehistorycn.php and php/_netvaluehistory.php to display all history data.
Jan 26, 2016
Add date display in related price time items following the advice from oldwain.
The original version omitted date display because I thougth it was obvious. And it is obvious too that not every one is as familar with both USA and China time as I am.
Although it is a small display change, I can not help myself to mofidy a lot of code.
The original StockReference class in /php/stock/stockref.php is now only as a base class for basic data and display data,
with SinaStockReference class and YahooStockReference class extends from it.
The original future data read becomes FutureReference class extends from StockReference,
and forex data read becomes ForexReference class extends from StockReference too.
In this way, the display is unified, so the new version number really has something new with it.
Original data member $strDate (2014-11-13, 'Y-m-d') and $strTime (08:55:00, 'H:i:s') in class StockReference are kept as the same,
added new data member $strDateTimeDisplay (11-13 08:55) for display, separating data and display.
Feb 26, 2016
SZ162411 is trading more than 10% higher than its net value recently, the max premium as high as 17%, so the XOP and SZ162411 arbitrage is not possible now.
Add XOP and two oil ETF USO/USL pair trading tool page.
The current day pair trading price uses the same method as the leverage ETF price estimation in the future ETF pages.
Continuing to organzie the similar code, extends MyStockReference class from SinaStockReference class,
and finally extends MyLeverageReference class from MyStockReference.
As we called mysql related function like SqlGetStockDescription of /php/sql/sqlstock.php in MyStockReference class,
To keep the independence of the different modules in /php, MyStockReference and MyLeverageReference class are put in a new file /php/mysqlstock.php.
March 11, 2016
As Chinese stock going down frequently, I feel more and more unsafe for SZ150022 now. Added Shenzhen Index A page to calculate it more carefully.
And added another similar graded fund H Shares A at the same time as well.
Different users have been suggesting to add estimation value in the history net value table of SZ162411.
Except for not willing to show my possible error directly, I did not add it because the change is realtime, and I don't know when to record it, after US market close or Chinese market close?
In the LOF code, the variable for estimation value was originally in _LofGroup class.
With my new _GradedFundGroup class in file php/_gradedfund.php having 3 class member of FundReference class in file /php/stock/fundref.php,
naturally I moved the estimation value to FundReference class. And when estimaation value and net value variable are put together, the data structure leads my mind again,
suddenly I realized that it is most reasonable to record the estimation value in the same time when the current day net value is recorded!
US Daylight Saving Time
March 14, 2016
A bug is found as US enter daylight saving time: date_default_timezone_set('EST') is not considering daylight saving,
need to use date_default_timezone_set('America/New_York') or date_default_timezone_set('EDT').
March 25, 2016
As Gold future GC is not trading on Easter holiday, I get the chance to adjust the net value estimation for Chinese Gold ETF,
including GuoTai Gold ETF, HuaAn Gold ETF, E Fund Gold ETF,
Bosera Gold ETF, Universal Gold LOF, Harvest Gold LOF and E Fund Gold LOF.
The average page view of my net value pages in a normal trading day is around 1000 now, the max day is nearly 1700. I have been optimizing my software for more page views in the future.
As the SMA caculation only need to be done once in a day, it is natural to save the result instead of calculate it with every page view.
I have been thinking about it but never did anything, until I finished the adjustment of 7 gold ETF, I realized that the SMA of GLD need to be calculated in 8 pages including the GC gold future page.
I feel I can not stand it any more.
Based on the experience of finding HTTPRequest and Mobile-Detect code on internet,
I underestimated the efforts to find a ready php class for config file read and write.
An easy found one cost 5 USD, it says to support file and mysql for configuration at the same time, as I do not like too many tables in mysql and only interested in file config,
I think 5 USD may be a waste of money.
As last I found the free INIFile, put it into /php/class/ini_file.php file.
The class only support config read and write on an existing file, still as a PHP new programmer, I spent a few hours to modify it and finally made it works with my pages.
Sina Realtime HK Stock Data
Apr 18, 2016
With the help of rdcheju,
now using Sina realtime HK stock data(http://hq.sinajs.cn/list=rt_hk02828) to replace original Sina HK data which has 15 minutes delay.
Apr 23, 2016
On last Thursday the total page view of my net value pages was over 2200, this encouraged me to do more page speed optimization.
Removed leverage ETF data from LOF pages, the old usage was moved to the SPY and SH/SDS,
VXX and XIV/UVXY pair trading page.
The Most Stupid Bug in Recent Years
May 15, 2016
Last week USDCNY was rising significantly again, and a new bug of SZ162411 net value estimation floated on water. On Friday, May 13, my estimation was about 1% higher than the official data.
I began to check the problem yesterday, found that the last automatic calibration was done on the evening of May 12, after the official net value of May 11 data released.
And the automatic calibration on May 13 was not done yet. In other words, the calibration was one day late for quite some time now, but when USDCNY had little change, the problem was ignored.
It was easy to find the bug, I used the simplest way to fix the estimation bug (mentioned in Sina fund data) caused by continue Chinese and USA market close after Chinese New Year.
Because only Hongkong LOF would have the chance of official LOF net value newer than ETF, I rewrote part of the code by checking if it was HK LOF,
and modified an actually unrelated code, what should be $iHours = STOCK_HOUR_END + ($this->usdhkd_ref ? 0 : 24); was written as $iHours = STOCK_HOUR_END + ($this->usdhkd_ref) ? 0 : 24;
But this bug made me feel very bad, this time I can not say I am still a 6 years new PHP programmer. As a proud 25 years C programmer, this is also a very stupid bug in C language!
June 5, 2016
Wang Xiaobo always says that he is writing his books using software editer programmed by himself. I was laughing at the idea 20 years ago.
To my own surprise, I began to write my own stock software after all those years. People think differently in different age.
When My Portfolio function first online, the page load speed was very slow,
and it will become much faster when refresh. Not confident about my mysql skills, I started optimizing database at once.
I solved some obvious problems. For example, I extended the data field of original stockgroupitem table, made necessary calculation for all records with the same groupitem_id in stocktransaction table,
and saved the result in stockgroupitem table, so to avoid query stocktransaction table and calculate every time.
However, after all those changes, the speed was still slow. But the good news is, after reviewing the software again and again, I finally found the real reason.
After get stock group from member_id in stockgroup table,
I will construct a MyStockGroup class instance for each stockgroup.
And in the original construct function of MyStockGroup class, it will construct a MyStockTransaction class instance for each stock in the stockgroup,
Because MyStockTransaction class need MyStockReference class as parameter,
if no existing MyStockReference instance for the stock, it will construct a new one.
The result was, when I was getting my portfolio in the first time, I would download nearly all stock data from Sina, no wonder it was so slow.
The problem is easy to solve, first I will not construct MyStockTransaction class instance if there is no stock transaction at all,
then I will group all stock when has transaction, and to prefetch all the data from Sina at the same time.
The prefetch code is put in new file /php/stock/stockprefetch.php.
Finally I used the prefetch method in all places when need to download data from Sina, including the SZ162411 net value calculation of course.
All page speed is improved in some way. It seems that I was wrong to explain why my net value pages were slow some time earlier,
I thought it was because the web server was in US and my major visitors were from China.
T+1 Estimation with Current CL Factor in
Aug 18, 2016
Many people's Excel sheet has this one, so I added it too.
Sep 18, 2016
Automatic and manual calibration history
Oct 6, 2016
SZ162411 calibration history.