本文目录🧾
正文部分📝
毕业设计-分别基于pyqt和django的桌面应用以及web应用的数据可视化系统
先上一个最终效果图:
所用工具
- Python 3.6
- SQLite 3
- QT 5.10.1
所用数据
大家自行在百度网盘下载:A股数据
数据处理
这个部分我已经在前面《A股资产负债比率与五大财务比率》博文中已经详细描述了,请大家移步查看。按上述教程获得后的数据(该数据不提供,请大家自己实验),如下图所示:
但是,几千张表直接导入数据库,明显是不合理的,所以我们要对这些表进行一些处理合成一张表,这样进行查询就方便许多了。我的方案是弄成三张表,一张是所有股票公司的资产负债率表,一张为所有股票公司的五大财务比率表,最后一张为资产负债率和五大财务比率都包括的表。这样设置的理由,我会在接下来的过程中说明。
这里我要强调一点:因为之前数据里是没有显示股票名称的,但我们查询某只股票的时候(想想你们大二学的数据库),那么我必须要有查询条件,所以我们要在数据里添加一列显示股票名称。比如我本来数据是下面这样:
这样我们无法知道每一行数据到底是哪知股票公司的,所以需要加上一列股票名称,就变成下图这样:
合并所有表格以及生成第一列的代码如下:
import numpy as np
import pandas as pd
import os,re,csv,math
from pandas.core.frame import DataFrame
path_root = r'G:\四年记\比赛\炒个股\五大财务比率'
alls=[]
stocks=[]
for file in os.listdir(path_root): #遍历目录里所有文件名
ls=file.strip('.csv').split('-')
print(ls[1]) #获得股票的名称
filename=os.path.join(path_root,file) #得到文件的地址
f1 = open(filename) #打开文件
df = pd.read_csv(f1) #获取数据
# 被注释部分的代码是获得数据第一列,对应的是股票名字。就是我上面讲的,我这里是单独生成一个csv,你们也可以对代码进行改进。
# df['stock_Name']=ls[1] #在df添加一列名字为stock_Name的列,保存股票公司名称。
# stock=np.array(df['stock_Name']) #将dataframe转成数组
# stock=stock.tolist() #将数组转成list
# for rowl in stock:
# stocks.append(rowl) #这样就能保存所有股票的名称在stocks里了
df=np.array(df) #这里是将数据转为数组
df=df.tolist() #将数组转为list
for row in df:
alls.append(row) #这样就能保存所有股票的数据在alls里了
# stocks.append(ls[1])
#stock=DataFrame(stocks)
#stock.to_csv(r'C:\Users\mac\Desktop\第一列.csv', sep=',',encoding='ANSI')
alldata=DataFrame(alls) #最后将list转成dataframe进行保存
alldata.to_csv(r'C:\Users\mac\Desktop\五大财务比率.csv', sep=',',encoding='ANSI')
将上述代码注释部分的#删掉,再将for循环语句里的非注释部分加上#就能保存第一列了。获得资产负债比率的话,只需改一下path_root即可。然后在excel里将第一列值复制到数据里即可。因为当时为了省时间就没有去合并dataframe了,直接在excel里操作,大家也可以自己合并下dataframe后再保存。因为数据里有空值,我们想在界面显示,就要对数据进行处理。这里就教大家一个excel快速填补空值的方法:Excel如何查找区域中的空白值并填充数据?,大家将数据补成Null即可。最后将csv文件保存为txt(制表符分割),再将txt转成utf-8格式,因为sqlite3数据库只能接收utf-8格式文件,否则会显示出错。
这部分有些难理解,所以我为大家提供处理后的三个表格数据,这里知道怎么去思考数据处理即可,数据下载链接:处理后的三表数据。
导入SQLite3数据
大家先行下载SQLite3和SQLitestudio,以防你们下错了,所以我也上传网盘了,下载链接:SQLite3。
下载完之后,解压studio,然后在studio里像使用sqlserver一样,建数据库和三张表,三个表的格式都一样,如下所示:
你们就按上面的格式来制表。将sqlite3.exe放到你数据库所在的目录里,制好表后,就点sqlite3.exe,然后按如何高效地向SQLite导入大量数据教程进行导入数据,记得修改分隔符,我们上面保存的分隔符是制表符分割,所以要修改为’\t’。所导入的数据就是我上面处理得到的三个表数据。
导入成功的话,是没有提示的,有提示的就代表出了错。导入成功后,可以使用sql语句进行查询操作,和SQLserver一样。
界面设计
界面设计我用的是QtDesigner,就在Qt里面。在做应用之前,为大家提供一个基础的范例:py应用范例,这里有涉及登陆注册功能,值得看看。
QtDesigner主界面如下图所示:
我一般选择Dialog without button,选项之间的差距大家自行百度。在界面左边有许多拖件,大家只需要将自己需要的插件拖到页面来即可,按自己喜好来。具体的各个插件的用处,请大家自行查询tutorial。一般用的多的是pushButton,tableWidget,LineEdit,Label这几个。我初始UI文件如下图:
这只是大致模型,到后面就会知道和ui文件有很大差距。接下来就是将ui文件转成我们熟悉好操作的py文件,先在python里安装pyuic5,然后使用以下指令:
pyuic5 -o 1.py 1.ui
转成py文件后,记得将报错的最后一行删除,然后在最后加上应用的主函数:
if __name__ == "__main__":
import sys
app = QtWidgets.QApplication(sys.argv)
Dialog = QtWidgets.QDialog()
ui = Ui_Dialog()
ui.setupUi(Dialog)
Dialog.show()
sys.exit(app.exec_())
在生成的py文件里有一个方法Setup(),可以看到里面有你在ui文件里拖曳的插件。另外一个方法是retranslateUi(),它是写界面初始显示的内容。
接下来就是对各个插件,进行一些美化,美化部分就不细说了,和CSS类似,不过这里需要调用setStyleSheet方法进行位置、大小、字体、颜色、背景图片的设置。具体的使用方法,可以参照文末给的项目源代码。
SQLite数据库与界面的连接
self.connection = sqlite3.connect("stock.db") #连接数据库
Data1=self.connection.execute("SELECT types,first,second,third,fourth,fifth FROM fuzhai where stockname= '国农科技'") #查询数据,查询语句就是SQL语句
self.connection.commit() #提交事务
Data1=Data1.fetchall() #将数据转成list
数据显示功能
其实就是将上一个步骤获得的数据插入到tableWidget(表格框架)即可:
self.tableWidget.setRowCount(11) #设置行数
self.tableWidget.setColumnCount(6) #设置列数
self.tableWidget.setHorizontalHeaderLabels(['资产负债比率','2017年','2016年','2015年','2014年','2013年']) #设置列名
self.tableWidget.verticalHeader().setVisible(False) #隐藏垂直header
self.tableWidget.horizontalHeader().setFixedHeight(32) #设置行高
self.tableWidget.setShowGrid(False) #隐藏网格线
for index in range(self.tableWidget.columnCount()): #设置列
headItem = self.tableWidget.horizontalHeaderItem(index)
headItem.setFont(QFont("微软雅黑", 9, QFont.Bold))
headItem.setForeground(QBrush(QColor( 0,0,0)))
headItem.setTextAlignment(Qt.AlignLeft | Qt.AlignVCenter)
self.tableWidget.setEditTriggers(QAbstractItemView.NoEditTriggers) #设置不能对网格数据进行修改
#以下是设置每列宽度,第一个数字是行号,第二个数字是宽度大小
self.tableWidget.setColumnWidth(0,175)
self.tableWidget.setColumnWidth(1,80)
self.tableWidget.setColumnWidth(2,80)
self.tableWidget.setColumnWidth(3,86)
self.tableWidget.setColumnWidth(4,88)
self.tableWidget.setColumnWidth(5,82)
#以下是设置每行高度,第一个数字是行号,第二个数字是高度大小
self.tableWidget.setRowHeight(0,28)
self.tableWidget.setRowHeight(1,28)
self.tableWidget.setRowHeight(2,28)
self.tableWidget.setRowHeight(3,28)
self.tableWidget.setRowHeight(4,28)
self.tableWidget.setRowHeight(5,28)
self.tableWidget.setRowHeight(6,28)
self.tableWidget.setRowHeight(7,28)
self.tableWidget.setRowHeight(8,28)
self.tableWidget.setRowHeight(9,28)
self.tableWidget.setRowHeight(10,28)
for i in range(11):
if i%2==0:
for j in range(6):
if j==0:
temp_data=Data1[i][j] #临时记录,不能直接插入表格
data=QTableWidgetItem(str(temp_data)) #转换后可插入表格
data.setBackground(QBrush(QColor(30,136,168)))#设置颜色
self.tableWidget.setItem(i,j,data)
if j==1:
temp_data=Data1[i][j] #临时记录,不能直接插入表格
data=QTableWidgetItem(str(temp_data)) #转换后可插入表格
data.setBackground(QBrush(QColor(186,145,50)))#设置颜色
self.tableWidget.setItem(i,j,data)
if j==2:
temp_data=Data1[i][j] #临时记录,不能直接插入表格
data=QTableWidgetItem(str(temp_data)) #转换后可插入表格
data.setBackground(QBrush(QColor(0,137,108)))#设置颜色
self.tableWidget.setItem(i,j,data)
if j==3:
temp_data=Data1[i][j] #临时记录,不能直接插入表格
data=QTableWidgetItem(str(temp_data)) #转换后可插入表格
data.setBackground(QBrush(QColor(181,68,52)))#设置颜色
self.tableWidget.setItem(i,j,data)
if j==4:
temp_data=Data1[i][j] #临时记录,不能直接插入表格
data=QTableWidgetItem(str(temp_data)) #转换后可插入表格
data.setBackground(QBrush(QColor(101,50,124)))#设置颜色
self.tableWidget.setItem(i,j,data)
if j==5:
temp_data=Data1[i][j] #临时记录,不能直接插入表格
data=QTableWidgetItem(str(temp_data)) #转换后可插入表格
data.setBackground(QBrush(QColor(38,135,133)))#设置颜色
self.tableWidget.setItem(i,j,data)
else:
for j in range(6):
temp_data=Data1[i][j] #临时记录,不能直接插入表格
data=QTableWidgetItem(str(temp_data)) #转换后可插入表格
data.setBackground(QBrush(QColor(42,46,50)))#设置颜色
self.tableWidget.setItem(i,j,data)
效果如图:
这里是资产负债数据。按同样的方法进行一系列修改,再对select语句修改,就能获得五大财务比率的数据,(具体见文末源代码里的tableWidget_2)如下:
股票查询功能
我在界面里设置了一个LineEdit插件和PushButton,来实现这个功能。其实主要的是获得用户在LineEdit输入的股票名称,然后在数据库里查询对应股票公司的数据,再进行上一个步骤将数据进行显示。
获取用户输入信息以及查询的代码如下:
stockname = self.textEdit.text()
self.connection = sqlite3.connect("stock.db")
Data1=self.connection.execute("SELECT types,first,second,third,fourth,fifth FROM fuzhai where stockname= '%s'"%stockname)
Data2=self.connection.execute("SELECT types,first,second,third,fourth,fifth FROM caiwu WHERE stockname = '%s'"%stockname)
self.connection.commit()
这里的pushButton调用search方法代码如下:
self.pushButton.clicked.connect(self.search)
具体的查看search()函数内的内容。
如果查询的股票在数据库中没有的话,会弹出一个警告,这段代码我写在warning.py文件里,要在stock.py里引入它的类名,这样才能在stock.py主文件里运行。
调用如下:
from warning import Ui_Dialog3
警告框其实就是新建一个Dialog,然后显示一张图片就好:
class Ui_Dialog3(object):
def setupUi(self, Dialog):
Dialog.setObjectName("Dialog")
Dialog.setFixedSize(310, 220)
Dialog.setStyleSheet("background-image:url(images/warning.png);\n")
Dialog.setWindowIcon(QtGui.QIcon('images/warning.ico'))
_translate = QtCore.QCoreApplication.translate
Dialog.setWindowTitle(_translate("Dialog", "warning"))
警告框如下图所示:
筛选股票功能
这是系统最重要的一part,先给各位科普一下intersection,是数据库里用来做交集的,大家之前应该没用到过,这在程序里起最重要的作用。
这里也顺便解释了上面的疑问——我为什么一定要做两表合起来的表。因为我们筛选一只股票,用于筛选的指标可能来自五大财务比率表也可能来自于资产负债比率表,如果我到两个表里分别去找符合的股票再来筛选会比较繁琐。所以这就是弄第三张表的原因所在。
我设置了十个指标,看下图即可明白:
具体指标的意义呢,我也解释不清楚,大家自己感兴趣就查查吧~筛选股票这里我设置的是QComboBox拖件,它可以设置多个选项,供用户选择。接下来po具体的查询符合条件的股票的代码:
#对获得的数据进行处理,得到可以放入查询语句里的纯数字。
zzczzl=self.zzczzl.currentText()
xjyydxj=self.xjyydxj.currentText()
xjyydxj=xjyydxj.strip('%').strip('>')
zzdays=self.zzdays.currentText()
zzdays=zzdays.strip('天').strip('<')
zcsyl=self.zcsyl.currentText()
zcsyl=zcsyl.strip('%').strip('>')
mll=self.mll.currentText()
mll=mll.strip('%').strip('>')
jll=self.jll.currentText()
jll=jll.strip('%').strip('>')
bjl=self.bjl.currentText()
bjl=bjl.strip('%').strip('>')
zzl=self.zzl.currentText()
zzl=zzl.strip('%').strip('>')
ztz=self.ztz.currentText()
ztz=ztz.strip('%').strip('>')
xjll=self.xjll.currentText()
xjll=xjll.strip('%').strip('>')
print(xjyydxj,zzdays,mll,jll,bjl,zzl,ztz,xjll)
if zzczzl=='<1':
print('hey')
zzczzl=zzczzl.strip('<')
print('zzczzl')
Data3=self.connection.execute("select stockname from two where types like '总资产周转率(次/年)' and first!='Null' and second!='Null' and third!='Null' and fourth!='Null' and fifth!='Null' and CAST(first as double)<'%s' and CAST(second as double)<'%s' and CAST(third as double)<'%s' intersect select stockname from two where types='现金与约当现金%s' and first!='Null' and second!='Null' and third!='Null' and fourth!='Null' and fifth!='Null' and CAST(first as double)>'%s' and CAST(second as double)>'%s' and CAST(third as double)>'%s' intersect select stockname from two where types like '应收款项周转天数(天)' and first!='Null' and second!='Null' and third!='Null' and fourth!='Null' and fifth!='Null' and CAST(first as double)<'%s' and CAST(second as double)<'%s' and CAST(third as double)<'%s' intersect select stockname from two where types like 'ROA=资产收益率%s' and first!='Null' and second!='Null' and third!='Null' and fourth!='Null' and fifth!='Null' and CAST(first as double)>'%s' and CAST(second as double)>'%s' and CAST(third as double)>'%s' intersect select stockname from two where types like '毛利率%s' and first!='Null' and second!='Null' and third!='Null' and fourth!='Null' and fifth!='Null' and CAST(first as double)>'%s' and CAST(second as double)>'%s' and CAST(third as double)>'%s' intersect select stockname from two where types='净利率%s' and first!='Null' and second!='Null' and third!='Null' and fourth!='Null' and fifth!='Null' and CAST(first as double)>'%s' and CAST(second as double)>'%s' and CAST(third as double)>'%s'intersect select stockname from two where types='经营安全边际率%s' and first!='Null' and second!='Null' and third!='Null' and fourth!='Null' and fifth!='Null' and CAST(first as double)>'%s' and CAST(second as double)>'%s' and CAST(third as double)>'%s' intersect select stockname from two where types='ROE=净资产收益率%s' and first!='Null' and second!='Null' and third!='Null' and fourth!='Null' and fifth!='Null' and CAST(first as double)>'%s' and CAST(second as double)>'%s' and CAST(third as double)>'%s' intersect select stockname from two where types='现金再投资比率' and first!='Null' and second!='Null' and third!='Null' and fourth!='Null' and fifth!='Null' and CAST(first as double)>'%s' and CAST(second as double)>'%s' and CAST(third as double)>'%s'intersect select stockname from two where types='现金流量比率' and first!='Null' and second!='Null' and third!='Null' and fourth!='Null' and fifth!='Null' and CAST(first as double)>'%s' and CAST(second as double)>'%s' and CAST(third as double)>'%s' "%(zzczzl,zzczzl,zzczzl,'(占总资产%)',xjyydxj,xjyydxj,xjyydxj,zzdays,zzdays,zzdays,'(%)',zcsyl,zcsyl,zcsyl,'(%)',mll,mll,mll,'(%)',jll,jll,jll,'(%)',bjl,bjl,bjl,'(%)',zzl,zzl,zzl,ztz,ztz,ztz,xjll,xjll,xjll))
else:
zzczzl=zzczzl.strip('>')
Data3=self.connection.execute("select stockname from two where types like '总资产周转率(次/年)' and first!='Null' and second!='Null' and third!='Null' and fourth!='Null' and fifth!='Null' and CAST(first as double)>'%s' and CAST(second as double)>'%s' and CAST(third as double)>'%s' intersect select stockname from two where types='现金与约当现金%s' and first!='Null' and second!='Null' and third!='Null' and fourth!='Null' and fifth!='Null' and CAST(first as double)>'%s' and CAST(second as double)>'%s' and CAST(third as double)>'%s' intersect select stockname from two where types like '应收款项周转天数(天)' and first!='Null' and second!='Null' and third!='Null' and fourth!='Null' and fifth!='Null' and CAST(first as double)<'%s' and CAST(second as double)<'%s' and CAST(third as double)<'%s' intersect select stockname from two where types like 'ROA=资产收益率%s' and first!='Null' and second!='Null' and third!='Null' and fourth!='Null' and fifth!='Null' and CAST(first as double)>'%s' and CAST(second as double)>'%s' and CAST(third as double)>'%s' intersect select stockname from two where types like '毛利率%s' and first!='Null' and second!='Null' and third!='Null' and fourth!='Null' and fifth!='Null' and CAST(first as double)>'%s' and CAST(second as double)>'%s' and CAST(third as double)>'%s' intersect select stockname from two where types='净利率%s' and first!='Null' and second!='Null' and third!='Null' and fourth!='Null' and fifth!='Null' and CAST(first as double)>'%s' and CAST(second as double)>'%s' and CAST(third as double)>'%s'intersect select stockname from two where types='经营安全边际率%s' and first!='Null' and second!='Null' and third!='Null' and fourth!='Null' and fifth!='Null' and CAST(first as double)>'%s' and CAST(second as double)>'%s' and CAST(third as double)>'%s' intersect select stockname from two where types='ROE=净资产收益率%s' and first!='Null' and second!='Null' and third!='Null' and fourth!='Null' and fifth!='Null' and CAST(first as double)>'%s' and CAST(second as double)>'%s' and CAST(third as double)>'%s' intersect select stockname from two where types='现金再投资比率' and first!='Null' and second!='Null' and third!='Null' and fourth!='Null' and fifth!='Null' and CAST(first as double)>'%s' and CAST(second as double)>'%s' and CAST(third as double)>'%s'intersect select stockname from two where types='现金流量比率' and first!='Null' and second!='Null' and third!='Null' and fourth!='Null' and fifth!='Null' and CAST(first as double)>'%s' and CAST(second as double)>'%s' and CAST(third as double)>'%s' "%(zzczzl,zzczzl,zzczzl,'(占总资产%)',xjyydxj,xjyydxj,xjyydxj,zzdays,zzdays,zzdays,'(%)',zcsyl,zcsyl,zcsyl,'(%)',mll,mll,mll,'(%)',jll,jll,jll,'(%)',bjl,bjl,bjl,'(%)',zzl,zzl,zzl,ztz,ztz,ztz,xjll,xjll,xjll))
中间那段查询语句看起来很多,其实理解了就很简单,因为之前我们设置了Null值,那些有Null值的股票公司不能进入筛选列表里,并且我们这里是五年数据,所以每一年都需要考虑满足所选的条件(我这里是算三年,五年同时满足的股票比较少)。最重要的就是使用intersection获取交集,得到满足全部条件的股票。
具体代码请查看getRecommend()函数。
显示资产负债比率变化趋势图功能
这里也设置了pushButton来获取图像,和上面说的类似,就不重复了。
图像绘制是使用matplotlib来进行画图,先在数据库调用数据,再将数据进行处理,最后进行绘制图像:
matplotlib.rcParams['font.family']='SimHei' #设置中文
db = sqlite3.connect("stock.db")
Data1=db.execute("SELECT types,first,second,third,fourth,fifth FROM fuzhai where stockname= '%s'"%stockname)
db.commit()
Data1=Data1.fetchall()
lists=[]
labels=[]
#资产负债比率图片
for line in Data1:
lists.append(line[1:])
labels.append(line[0].replace('(占总资产%)','') ) #将一些不必要的文字删除,减少图片篇幅
#print(lists)
#print(labels)
#print(labels)
years=[2017,2016,2015,2014,2013]
i=0
fig=plt.figure(0)
global flag
flag=1
for one_list in lists:
if 'Null' in one_list:
flag=0 #有空值则返回,不绘制图像
return 0
#print(flag)
if flag==1:
for one_list in lists:
one_list=[float(one) for one in one_list]
#print(one_list)
plt.plot(years,one_list,"^-",label='%s'%labels[i])
i=i+1
plt.title('%s-资产负债比率'%stockname) #设置title
plt.ylim((-10,105)) #设置y轴
plt.yticks([])
plt.legend(loc='right',bbox_to_anchor=(0.2,0.82),ncol=1,fancybox=True)
plt.yticks([]) #设置纵坐标的值,因为值很杂,就设置不显示。
plt.xticks(np.linspace(2013,2017,5)) #设置横坐标的值
plt.savefig('images/pic.png',dpi=78)
plt.close(0)
return 1
绘制的图如下所示:
我又写了一个方法,来对该图像放大显示,在bigger.py里,在stock.py中对bigger.py进行了引入,在代码最前面。
导入的是类名哟~
from bigger import Ui_Dialog2
如果股票数据有Null值,就会显示一张提示图,该功能代码如下:
class Ui_Dialog2(object):
def setupUi(self, Dialog,flag):
Dialog.setObjectName("Dialog")
Dialog.setFixedSize(505, 384)
if flag==1:
Dialog.setStyleSheet("background-image:url(images/pic.png);\n")
else:
Dialog.setStyleSheet("background-image:url(images/null.png);\n")
Dialog.setWindowIcon(QtGui.QIcon('images/bigger.ico'))
_translate = QtCore.QCoreApplication.translate
Dialog.setWindowTitle(_translate("Dialog", "资产负债比率趋势图"))
效果图:
查询成功显示对应的变化趋势图:
查询的数据有Null值时的提示:
完整代码
源代码包:A股数据分析系统源代码
直接运行exe包:A股数据分析系统V1.0
(打包后的文件非常大,建议大家直接下载上面源代码,在有python环境的电脑上点击stock.py即可运行。)
尾语
毕业设计是想设计一个桌面系统应用以及一个线上web系统,但由于自己时间安排不妥,目前只搭建完了本地的桌面应用。线上基于Django的web应用目前只搭建好了框架,并实现了登陆注册功能。计划在之后的半个月里把基于Django的web应用完善好,完善计划如下:
- 实现搜索股票获得对应股票的资产负债比率表和五大财务比率表
- 利用echarts对数据进行多种可视化展示,如圆饼图,折线图,柱状图,dashboard图等
- 实现推荐界面,为用户推荐一些比较优质推荐入手的股票
- 根据各项指标,添加对股票对应公司的一些评价
- 在每只股票下面,加入评论系统
- 首页对所有股票分类显示,以便于查找感兴趣的股票