作者归档:bianbian

[原] IDL关于数字值域的注意点

IDL默认方式的数字是短整型(16-bit int)。所以如下的声明方式其实是报错的:

因为,1024*32=32768,超过了短整型上限(32767),从二进制码基础可知会跳到该类型的另一头,即:

这时有两种解决办法,一种显示声明为Long(32-bit int),即数字后面跟上L:

另外一种事先声明:

idl2参见文档,记得是编译开关:1)把所有数字作为Long型 2)用中括号表示数组下标,即[]
http://www.exelisvis.com/docs/COMPILE_OPT.html

[原] 根据自定义http header控制nginx选择反向代理服务器

好吧。本来想多写点的,现在没时间,太监了。
说下核心:客户端自定义的http header,在nginx的配置文件里能直接读取到。
条件:header必须用减号“-”分隔单词,nginx里面会转换为对应的下划线“_”连接的小写单词。
nginx配置:

测试:

[原] IDL的逻辑判断符和位操作运算符(坑啊)

好吧,又一次被IDL坑了。怪我自己没看仔细,主要语言太多。。。
比如:
C系列的逻辑判断符是 && || !
Delphi的逻辑判断符是AND OR NOT
写在这里,起警示作用:
IDL的逻辑判断符是这样的:&& || ~
IDL的位操作符是这样的:AND OR NOT XOR
以前写判断都是 if … and … then …
所以从来没注意and原来是(Bitwise AND),not原来是(Bitwise NOT)
好吧,因为以前的判断基本都是逻辑组合(即 if ID gt 2 and ID lt 5 then…)为主,所以基本两个的结果是一样的,没出过问题。
坑来了:
id = -2
if not id then …
(not -2) equals 1
所以,以后IDL逻辑判断全部要用 && || ~

[原] Installing Nagios 4.0.5 on nginx and Ubuntu 12.04

Nagios® Core™ is an Open Source system and network monitoring application. It watches hosts and services that you specify, alerting you when things go bad and when they get better.Nagios Core was originally designed to run under Linux, although it should work under most other unices as well.

以下是2014-4-17安装记录:

== 服务端 ==

安装依赖包:(其实不装也能安装成功,就是部分功能会disabled)

用户组:

从 http://www.nagios.org/download/ 下载解压:

编译安装:

建立htpasswd.users,这个如果装了apache2有htpasswd这个命令,不然可以用某个python脚本:
(如果服务器不想装python,可以找台有python的服务器;或者找台装了apache的机器运行htpasswd;或者装个httpd不启动也没关系)
(生成的htpasswd.users是个文本文件,复制到服务器的 /etc/nagios/htpasswd.users)

修改联系人email,在contacts.cfg的email那里(34行)

安装mail报警通知所需包(安装时选择”Internet Site”;输入域名)
(注意:不装这个mail包编译nagios-plugins-2.0.1会error,但是编译nagios-plugins-2.0没问题。。。)

安装插件:

启动nagios

安装fcgi

编辑nginx.conf

其中对/nagios/开头的地址进行了rewrite,(nagios内部很多页面是写死/nagios/开头的,导致images、css等出不来)

启动服务

登录后就看到nagios的监控界面了

[原] OpenSSL在多线程环境下的使用(避免core dump)

OpenSSL can safely be used in multi-threaded applications provided that at least two callback functions are set, locking_function and threadid_func.
OpenSSL 是线程安全的,前提是必须注册两个回调函数。其中根据 Openssl 的版本不同,会有不同 版本的 threadid 回调函数。
如果不注册回调函数,多线程下压力一大必挂无疑。基本都是core dump:
libcrypto.so
下面贴主要代码,如何注册使用这两个回调函数:
.h

.c
我这里用的是libuv,所以代码里是uv_mutex_t*,如果是pthreads,就换成 pthread_mutex_t*;相应的 uv_thread_self() 换成 pthread_self()

打开Trace日志,果然一堆lock的坑啊。。。。

… ssl locked: [type] 16, [file] ssl_lib.c, [line] 512
… ssl unlock: [type] 16, [file] ssl_lib.c, [line] 512
… ssl locked: [type] 2, [file] ex_data.c, [line] 304
… ssl unlock: [type] 2, [file] ex_data.c, [line] 325
… ssl locked: [type] 2, [file] ex_data.c, [line] 500
… ssl unlock: [type] 2, [file] ex_data.c, [line] 511

简单看了下OpenSSL的基本算法源码,可能OpenSSL当年实现的时候内存比较精贵,所以很多算法都是static的内存块,不停new free内存,所以多线程下必须用各种lock。
内存如此便宜的现如今,这种实现方法显然不符合多线程、高并发的潮流了。。。

参考:
http://www.openssl.org/docs/crypto/threads.html
http://blog.csdn.net/yasi_xi/article/details/19125103

[原] Nagios-xi在CentOS的安装tips (web server是nginx)

1)./fullinstall会报各种包依赖错误。(其实是 ./1-prereqs 这一步)
需要先卸载原来的mysql、php,反正哪些包依赖错误就卸掉原来的。

2)Nagios默认的httpd是apache,所以默认安装各种出错。修改 ./xi-sys.cfg

把 httpd='httpd'改成 httpd='nginx'

3)./A-subcomponents
Nagios的subcomponents里面很多install / post-install脚本还是写死的
service httpd restart,要改成 service $httpd restart
(nagioscore、nrdp),nagiosmobile是php写的安装脚本,不改没关系(不影响结果,不过会执行不下去)
反正按subcomponents的install脚本照着来,如果每个都确认解压或者复制成功,不用重头执行A-subcomponents,
直接 touch installed.subcomponents 算了

4)./B-installxi
跟上面3)的问题一样,httpd 改成 $httpd

5)./E-importnagiosql
因为apache的conf和nginx完全不同(而前面步骤Nagios装的是apache的conf),这一步需要访问
http://localhost/nagiosql/index.php 肯定出错,导致这一步出问题。
在/etc/nginx/nginx.conf内新增一行:
include /etc/nginx/conf.d/nagiosxi.conf;
在/etc/nginx/conf.d/下新建nagiosxi.conf,参照/etc/httpd/conf.d/nagiosql.conf设置:

(待续)

[原] 看好Microsoft .NET Native

.NET Native compiles C# to native machine code that performs like C++. You will continue to benefit from the productivity and familiarity of the .NET Framework with the great performance of native code. Popular Windows Store apps start up to 60% faster and use 15-20% less memory when compiled with .NET Native.
链接在此:Microsoft .NET Native

[原] netty 4.0.17.Final使用epoll模式的几个bug

netty支持linux的epoll是4.0.17.Final引入的新功能,不过测试使用发现几个bug。暂记如下:

[已解决的]
使用epoll时channel关闭错误

原因:Native.finishConnect()中有bug
见这里:https://github.com/netty/netty/issues/2280
下载最新代码编译package可解决。包括编译Native的c代码:

[未解决的]
使用epoll时channel中取不到localAddress()和remoteAddress()

log:

github上有相关的issue: https://github.com/netty/netty/issues/2262
虽然标记为已解决,但编译4.0最新的代码(4.0.18.Final-SNAPSHOT)错误仍在。
先记录下,明天再来看怎么回事。

[原] 使用nodejs判断断线并重启Netgear路由器(电信光纤无故中断后)

家里宽带是电信的100M光纤。不过有时候会莫名其妙断线,而且只能通过重启路由器来解决。(10000反映多次未果。。。)
于是没办法,写了个nodejs脚本(这是我第一个nodejs程序好吗)来判断是否断线并重启路由器。
本文所用路由器型号为Netgear 6300 v2
主要思路:
检查 http://10.0.0.1/ADV_home2.htm
根据内容里的:Domain Name Server 是否存在有效的DNS来判断是否断线
如果断线,则提交相应的数据(buttonSelect=2)使Netgear路由器重启

[原] 如何在数组中查找定位最接近的数值

这个需求一般是查询一个look-up-table,比如数据定标表,假设有如下一个数组(实际数组比这大很多,只是为了举例):
现在要找 1.5 最接近的数值:

我一开始用的是先把数组sort,然后用二分法查找(代码不贴了,很简单)
后来一想IDL应该有内置支持这种查找吧,发现有 Value_Locate 方法,可以解决问题。
但是缺点很明显,需要先sort
后来找到这里:Locating a Value in an Array
我稍微改进了一下,因为Min支持 /ABSOLUTE 参数:

[转] 如何高效地编写IDL代码 Tips & Thicks

适合所有IDL使用者好好看看:
Tips & Tricks for Efficient IDL Programming
尤其是3、5、9、10条:
3、Try to avoid IF statements within loops. When an IF statement appears in the middle of a loop with each element of an array in a conditional, the loop can be eliminated most of the time by using logical array expressions.

Slow:
for i = 0L, n_elements(data)-1L do $
if (data[i] le 30) then $
data[i] = 30

Faster:
dex = where(data le 30, count)
if (count gt 0) then data[dex] = 30

Even faster:
data = ((data gt 30) * data) + ((data le 30) * 30)

Fastest:
data = data > 30

5、Access large arrays by memory order. Arrays in IDL are column-major order. The important thing to remember is that IDL indexes arrays as [column, row]. The upper left-hand element of a matrix is considered to be [0,0]. This is the format used by FORTRAN, and is traditionally associated with image processing because it keeps all the elements of a single image scanline together. In contrast C and Visual Basic use row-major order and indexes its data as [row,column].

When an array is larger than or close to the working set size (the amount of physical memory available for the process) it should be accessed in memory order, meaning access it by accessing rows first.
for x = 0, 511 do for y = 0, 511 do arr[x, y] = ….
is very inefficient because to read the first column 250,000 bytes of data must be read into physical memory. This process must be repeated for each column, requiring the entire array to be read and written almost 512 times. By exchanging the two FOR loops the computing time can be reduced by a factor of 50:
for y = 0, 511 do for x = 0, 511 do arr[x, y] = ….

9. Use of the TEMPORARY function minimizes memory use when performing operations on large arrays. TEMPORARY reassigns to a new variable the memory referred to by its argument, deleting the old variable reference in the process (though the reassignment may be to a variable that reuses the original name). Assume that A is a large array. To add 1 to each element of A this is one coding option:
A = A + l
This statement creates a new array for the result of the addition and assigns the result to A before freeing the old allocation of A. Therefore, this operation needs 2*sizeof(A) of memory to perform the operation. The statement
A = temporary(A) + 1
needs no additional space.

10. Try not to use “*” for array indexing on the left hand side of a statement. Instead, use “0.” For instance, for the array
B = intarr(200, 200, 3)
it is much slower to use the following notation
B[*,*,1] = insertData
than to use
B[0,0,1] = insertData
The first notation is inefficient because the IDL interpreter will allocate/build a 200 x 200 long integer array of subscript indexes to substitute for the wildcard token. The latter notation does not need an array of subscripts; it performs direct copy to memory. (Note that ‘insertData’ must fit in the 200 x 200 x 2 memory space that defines the borders of the subarray that starts at ‘B[0,0,1]’ in order to avoid an array-out-of-bounds error.)

[哎] 再吐个槽,bianbian.org域名恢复了

好吧,费尽千辛万苦,寄了各种材料,总算把域名转出来了。
抱歉没有时间精力通过申诉途径拿到转移密码,被edong.com坑去50元资料修改费,另外快递费44。
即使这样,也搞了一个月时间。。。。。。

[哎] 吐个槽,blog恢复了

感谢郭嘉,自动linode被封以后blog就无法访问了,一直懒得管它。
最近搬到amazon的aws上,总算恢复了一大半。不过bianbian.org域名还是感谢郭嘉无法及时转出。
要完全正常再等等吧。。。
其实这篇主要不是来吐槽的。
自从土豪一把上了 Netgear R6300 v2 以后,以前的路由器 tp-link WR741N 就闲置下来,然后今天抽经就折腾了一把。
刷Gargoyle,ssh到路由器上,opkg install一堆感谢郭嘉的组件后,总算在路由器上实现了自动proxy。
妈妈再也不用担心手机iPad要用VPN才能感谢郭嘉了。。。
几个有用的链接:OpenWRTGargoyleshadowsocks-on-openwrtPrivoxyport-mirroring

[原] Delphi加密解密控件DCPcrypt的RC4实现bug

作者David Barton网站:http://www.cityinthesky.co.uk/opensource/dcpcrypt
DCPcrypt v2 contains full implementations of the following algorithms:
Encryption Algorithms Hash Algorithms
Blowfish Haval
Cast 128 MD4
Cast 256 MD5
DES, 3DES RipeMD-128
Ice, Thin Ice, Ice2 RipeMD-160
IDEA SHA-1
Mars SHA-256, SHA-384, SHA-512
Misty1 Tiger
RC2, RC4, RC5, RC6
Rijndael (the AES)
Serpent
Tea
Twofish
不过我试了最新的控件,RC4实现还是有bug:8192字节后,加密的数据不是标准的。(和C语言比)
如果你全都用这个控件来实现RC4的加密/解密,或者数据块不超过8192字节,那没有问题;
不过如果你跨语言了(即用Delphi加密,其他语言解密;或者其他语言加密,Delphi解密),而且数据块超过8192字节,那一定会遇到这个bug:8192字节后的加密/解密内容不匹配。

【补充】经过阅读代码,我发现了这个bug的原因:
在 DCPcrypt2.pas 中, 当调用 TDCP_cipher.EncryptStream 时,作者把Stream 按照8192字节分成几部分,然后对每一部分调用Encrypt函数。问题就出在这里,因为RC4的算法,KeyData[0..255]这个数组是不断在变化的。作者分块以后,调用Encrypt()偏移量又重新从下标0开始算。就会导致问题:

[原] Android Activity的onConfigurationChanged不被执行

好吧,第一次做Android应用,先吐个槽,感觉xml和java代码切换来切换去的挺麻烦的。
由于要判断ORIENTATION_PORTRAIT(直板方向)和ORIENTATION_LANDSCAPE(平板方向),于是按照网上的普遍教程,在Activity的xml里设置
android:configChanges="orientation|keyboardHidden",在java里代码判断:

不过,没有用,onConfigurationChanged并没有被执行。于是继续google,发现这里:
onconfigurationchanged-not-getting-called
原来是这样:Caution: Beginning with Android 3.2 (API level 13), the “screen size” also changes when the device switches between portrait and landscape orientation. Thus, if you want to prevent runtime restarts due to orientation change when developing for API level 13 or higher (as declared by the minSdkVersion and targetSdkVersion attributes), you must include the “screenSize” value in addition to the “orientation” value. That is, you must decalare android:configChanges=”orientation|screenSize”. However, if your application targets API level 12 or lower, then your activity always handles this configuration change itself (this configuration change does not restart your activity, even when running on an Android 3.2 or higher device).
(From http://developer.android.com/guide/topics/resources/runtime-changes.html)
即从API 13开始,要设置为:android:configChanges="orientation|keyboardHidden|screenSize"