Archive

Archive for November, 2018

解决Python Error ‘TSaslClientTransport’ object has no attribute ‘trans’

November 12th, 2018 No comments

解决Python Error ‘TSaslClientTransport’ object has no attribute ‘trans’
原因应该是thrift和impyla包版本的问题

sudo pip uninstall thrift
sudo pip uninstall impyla
sudo pip install thrift==0.9.3
sudo pip install impyla==0.13.8

参考
http://community.cloudera.com/t5/Interactive-Short-cycle-SQL/Python-Error-TSaslClientTransport-object-has-no-attribute/m-p/58033

Categories: 系统管理 Tags: ,

RQAlpha Docker容器化Dockerfile

November 12th, 2018 No comments

最近需要使用RQAlpha进行国内的量化分析工作,并对RQAlpha进行了容器化处理,
方便使用docker和ks进行容器化管理

1.基于 jupyter/minimal-notebook jupyter的官方镜像,默认使用python3
2.使用Anaconda的mini-conda进行环境管理,可以自由切换python27和python3运行环境
3.使用pip安装了其他非conda管理的包内容

pip install bs4 cx-Oracle docopt future hdfs pyecharts PyMySQL raven typing lxml

4.使用编译方式安装了TA-lib,暂时为32bit版本,安装了国内流行的tushare

pip install tushare TA-lib

5.将matplotlib.pyplot自动导入,同时配合安装了国内流行的pyechart
6.安装了HDFS,cx-Oracle,PyMySQL等常用的数据连接包

可以在以下位置找到
https://github.com/limccn/rqalpha/tree/master/docker

Read more…

Categories: 系统管理 Tags: ,

RQAlpha BUG Issue#219

November 12th, 2018 No comments

Pingback https://github.com/ricequant/rqalpha/issues/219

Hello, RQAlpha Team

RQAlpha is really a effective tool for price back-testing.

I found something wrong when using command `# rqalpha plot someresult.pkl` to plot my back-testing result. It came out a blank window. I tried to solve this problem and found something interesting.

in `rqalpha/rqalpha/mod/rqalpha_mod_sys_analyser/plot.py`, line 52-53

 portfolio = result_dict["portfolio"]
 benchmark_portfolio = result_dict.get("benchmark_portfolio")

 print portfolio.index
 print benchmark_portfolio.index

by printing portfolio.index and benchmark_portfolio.index , I found an unreasonable difference.

![image](https://user-images.githubusercontent.com/4476941/33165321-59a13c02-d071-11e7-80e1-b41a5fbdd6db.png)

According to https://github.com/pandas-dev/pandas/issues/8614 says, matplotlib can not plotting when DatetimeIndex is created by pandas > 0.15 .

I found a temporary way to solve this problem. and finally plotting was working functionally.

Use `index.to_pydatetime()` to explicitly convert `DatetimeIndex` type index to Python `Datetime` type.

For example: modify `rqalpha/rqalpha/mod/rqalpha_mod_sys_analyser/plot.py`, line 152
`ax.plot(portfolio[“unit_net_value”] – 1.0, label=_(u”strategy”), alpha=1, linewidth=2, color=red)`
to
`ax.plot(index.to_pydatetime(),portfolio[“unit_net_value”] – 1.0, label=_(u”strategy”), alpha=1, linewidth=2, color=red)`

Although it is not the best way to solve this problem, I know you can find the best one finally.
And I hope these information may help you.

Thanks.

Categories: 算法研究, 系统管理 Tags:

PostgreSQL查询表和index占用空间大小

November 12th, 2018 No comments

PostgreSQL查询表和index占用空间大小

PostgreSQL表和index占用空间大小信息存储在
information_schema.tables中
通过SQL可以查询到相应的统计数据

--查出单个表的大小
SELECT pg_size_pretty(pg_relation_size('TABLENAME'));

查出表大小按大小含Index

-- 查出表大小按大小含Index
SELECT
"table_name",
pg_size_pretty(table_size) AS table_size,
pg_size_pretty(indexes_size) AS indexes_size,
pg_size_pretty(total_size) AS total_size
FROM (
SELECT
TABLE_NAME,
SUBSTRING("table_name",1,10) AS short_name,
pg_table_size(TABLE_NAME) AS table_size,
pg_indexes_size(TABLE_NAME) AS indexes_size,
pg_total_relation_size(TABLE_NAME) AS total_size
FROM (
SELECT ('"' || table_schema || '"."' || TABLE_NAME || '"') AS TABLE_NAME
FROM information_schema.tables
) AS all_tables
WHERE all_tables.table_name LIKE '%TABLENAME%'
ORDER BY total_size DESC
) AS pretty_sizes

Read more…

Nginx HTTP站点使用301跳转HTTPS

November 12th, 2018 No comments

方法一,使用Return方式,适用于全站HTTPS

server_name  www.lidaren.com lidaren.com;
return       301 https://www.lidaren.com$request_uri;

方法二,rewirte方式转发特定目录,适用于子目录HTTPS化
location 使用 ·/· 根目录则全站跳转
location 使用 ·/XXX目录· 子目录则跳转指定子目录

location / {
    rewrite ^(.*) https://www.lidaren.com$1 permanent
}

完整参考

server {
    listen       80;
    server_name  www.lidaren.com lidaren.com;
 
# 方法一,使用Return方式,适用于全站HTTPS
    return       301 https://www.lidaren.com$request_uri;
 
# 方法二,rewirte方式转发特定目录,适用于子目录HTTPS化
# location 使用/根目录则全站跳转
#    location / {
#        rewrite ^(.*) https://www.lidaren.com$1 permanent
#    }
}
Categories: 系统管理, 零敲碎打 Tags:

Python 代码方式生产中国身份证号码

November 9th, 2018 No comments

Python 代码方式生产中国身份证号码
python做爬虫需要填写中国身份证号码,而且又各种验证规则,所以网上找了个靠谱的

import random, datetime
 
def ident_generator():
    #身份证号的前两位,省份代号
    sheng = ('11', '12', '13', '14', '15', '21', '22', '23', '31', '32', '33', '34', '35', '36', '37', '41', '42', '43', '44', '45', '46', '50', '51', '52', '53', '54', '61', '62', '63', '64', '65', '66')
 
    #随机选择距离今天在7000到25000的日期作为出生日期(没有特殊要求我就随便设置的,有特殊要求的此处可以完善下)
    birthdate = (datetime.date.today() - datetime.timedelta(days = random.randint(7000, 25000)))
 
    #拼接出身份证号的前17位(第3-第6位为市和区的代码,中国太大此处就偷懒了写了定值,有要求的可以做个随机来完善下;第15-第17位为出生的顺序码,随机在100到199中选择)
    ident = sheng[random.randint(0, 31)] + '0101' + birthdate.strftime("%Y%m%d") + str(random.randint(100, 199))
 
    #前17位每位需要乘上的系数,用字典表示,比如第一位需要乘上7,最后一位需要乘上2
    coe = {1: 7, 2: 9, 3: 10, 4: 5, 5: 8, 6: 4, 7: 2, 8: 1, 9: 6, 10: 3, 11:7, 12: 9, 13: 10, 14: 5, 15: 8, 16: 4, 17: 2}
    summation = 0
 
    #for循环计算前17位每位乘上系数之后的和
    for i in range(17):
        summation = summation + int(ident[i:i + 1]) * coe[i+1]#ident[i:i+1]使用的是python的切片获得每位数字
 
    #前17位每位乘上系数之后的和除以11得到的余数对照表,比如余数是0,那第18位就是1
    key = {0: '1', 1: '0', 2: 'X', 3: '9', 4: '8', 5: '7', 6: '6', 7: '5', 8: '4', 9: '3', 10: '2'}
 
    #拼接得到完整的18位身份证号
    return ident + key[summation % 11]
 
ident_generator()

http://www.51testing.com/html/12/15124112-3705453.html

Categories: 语言编程, 零敲碎打 Tags:

npm中Error: could not get uid/gid问题的解决方法

November 9th, 2018 No comments

在Docker中运行npm出现Error: could not get uid/gid的问题

但是通过设置`unsafe-perm true` 可以解决这个问题,不可思议。

npm config set unsafe-perm true

其他可以用的npm配置
# 关闭安全证书检查

npm config set unsafe-perm true

# 关闭强制SSL

npm config set strict-ssl false

# 更改npm源

npm config set registry https://registry.npm.taobao.org

#设置代理

# socks5 proxy
npm config set proxy http://xx@xx.com:xxxx 
# http proxy
npm config set https-proxy=http://xx@xx.com:xxxx

参考
http://www.cnblogs.com/liyongjian5179/p/9884944.html

Categories: 系统管理 Tags: ,