• 关于

    array_work

    的搜索结果

问题

RAY Python框架内存不足

祖安文状元 2020-02-21 17:36:13 3 浏览量 回答数 1

回答

For small problems involving C code, it is often easy enough to use the ctypes modulethat is part of Python’s standard library. In order to use ctypes, you must first makesure the C code you want to access has been compiled into a shared library that iscompatible with the Python interpreter (e.g., same architecture, word size, compiler,etc.). For the purposes of this recipe, assume that a shared library, libsample.so, hasbeen created and that it contains nothing more than the code shown in the chapterintroduction. Further assume that the libsample.so file has been placed in the samedirectory as the sample.py file shown next.To access the resulting library, you make a Python module that wraps around it, suchas the following:# sample.pyimport ctypesimport os Try to locate the .so file in the same directory as this file_file = ‘libsample.so'_path = os.path.join(*(os.path.split(file)[:-1] + (_file,)))_mod = ctypes.cdll.LoadLibrary(_path) int gcd(int, int)gcd = _mod.gcdgcd.argtypes = (ctypes.c_int, ctypes.c_int)gcd.restype = ctypes.c_int int in_mandel(double, double, int)in_mandel = _mod.in_mandelin_mandel.argtypes = (ctypes.c_double, ctypes.c_double, ctypes.c_int)in_mandel.restype = ctypes.c_int int divide(int, int, int *)_divide = _mod.divide_divide.argtypes = (ctypes.c_int, ctypes.c_int, ctypes.POINTER(ctypes.c_int))_divide.restype = ctypes.c_int def divide(x, y): rem = ctypes.c_int()quot = _divide(x, y, rem) return quot,rem.value void avg(double *, int n)# Define a special type for the ‘double *‘ argumentclass DoubleArrayType: def fromparam(self, param):> typename = type(param).nameif hasattr(self, ‘[from](#)‘ + typename): return getattr(self, ‘from_‘ + typename)(param) elif isinstance(param, ctypes.Array):return paramelse:raise TypeError(“Can't convert %s” % typename)> # Cast from array.array objectsdef from_array(self, param): if param.typecode != ‘d':raise TypeError(‘must be an array of doubles')> > ptr, _ = param.buffer_info()return ctypes.cast(ptr, ctypes.POINTER(ctypes.c_double)) Cast from lists/tuplesdef from_list(self, param): val = ((ctypes.c_double)len(param))([](#)param)return val from_tuple = from_list Cast from a numpy arraydef from_ndarray(self, param): return param.ctypes.data_as(ctypes.POINTER(ctypes.c_double)) DoubleArray = DoubleArrayType()_avg = _mod.avg_avg.argtypes = (DoubleArray, ctypes.c_int)_avg.restype = ctypes.c_double def avg(values):return _avg(values, len(values)) struct Point { }class Point(ctypes.Structure): fields = [(‘x', ctypes.c_double),(‘y', ctypes.c_double)] double distance(Point *, Point *)distance = _mod.distancedistance.argtypes = (ctypes.POINTER(Point), ctypes.POINTER(Point))distance.restype = ctypes.c_double If all goes well, you should be able to load the module and use the resulting C functions.For example: >>> import sample >>> sample.gcd(35,42) 7 >>> sample.in_mandel(0,0,500) 1 >>> sample.in_mandel(2.0,1.0,500) 0 >>> sample.divide(42,8) (5, 2) >>> sample.avg([1,2,3]) 2.0 >>> p1 = sample.Point(1,2) >>> p2 = sample.Point(4,5) >>> sample.distance(p1,p2) 4.242640687119285 >>> 讨论 There are several aspects of this recipe that warrant some discussion. The first issueconcerns the overall packaging of C and Python code together. If you are using ctypesto access C code that you have compiled yourself, you will need to make sure that theshared library gets placed in a location where the sample.py module can find it. Onepossibility is to put the resulting .so file in the same directory as the supporting Pythoncode. This is what’s shown at the first part of this recipe—sample.py looks at the filevariable to see where it has been installed, and then constructs a path that points to alibsample.so file in the same directory.If the C library is going to be installed elsewhere, then you’ll have to adjust the pathaccordingly. If the C library is installed as a standard library on your machine, you mightbe able to use the ctypes.util.find_library() function. For example: >>> from ctypes.util import find_library >>> find_library('m') '/usr/lib/libm.dylib' >>> find_library('pthread') '/usr/lib/libpthread.dylib' >>> find_library('sample') '/usr/local/lib/libsample.so' >>> Again, ctypes won’t work at all if it can’t locate the library with the C code. Thus, you’llneed to spend a few minutes thinking about how you want to install things.Once you know where the C library is located, you use ctypes.cdll.LoadLibrary()to load it. The following statement in the solution does this where _path is the fullpathname to the shared library: _mod = ctypes.cdll.LoadLibrary(_path) Once a library has been loaded, you need to write statements that extract specific sym‐bols and put type signatures on them. This is what’s happening in code fragments suchas this: int in_mandel(double, double, int)in_mandel = _mod.in_mandelin_mandel.argtypes = (ctypes.c_double, ctypes.c_double, ctypes.c_int)in_mandel.restype = ctypes.c_int In this code, the .argtypes attribute is a tuple containing the input arguments to afunction, and .restype is the return type. ctypes defines a variety of type objects (e.g.,c_double, c_int, c_short, c_float, etc.) that represent common C data types. Attach‐ing the type signatures is critical if you want to make Python pass the right kinds ofarguments and convert data correctly (if you don’t do this, not only will the code notwork, but you might cause the entire interpreter process to crash).A somewhat tricky part of using ctypes is that the original C code may use idioms thatdon’t map cleanly to Python. The divide() function is a good example because it returnsa value through one of its arguments. Although that’s a common C technique, it’s oftennot clear how it’s supposed to work in Python. For example, you can’t do anythingstraightforward like this: >>> divide = _mod.divide >>> divide.argtypes = (ctypes.c_int, ctypes.c_int, ctypes.POINTER(ctypes.c_int)) >>> x = 0 >>> divide(10, 3, x) Traceback (most recent call last): File "<stdin>", line 1, in <module> ctypes.ArgumentError: argument 3: <class 'TypeError'>: expected LP_c_int instance instead of int >>> Even if this did work, it would violate Python’s immutability of integers and probablycause the entire interpreter to be sucked into a black hole. For arguments involvingpointers, you usually have to construct a compatible ctypes object and pass it in likethis: >>> x = ctypes.c_int() >>> divide(10, 3, x) 3 >>> x.value 1 >>> Here an instance of a ctypes.c_int is created and passed in as the pointer object. Unlikea normal Python integer, a c_int object can be mutated. The .value attribute can beused to either retrieve or change the value as desired. For cases where the C calling convention is “un-Pythonic,” it is common to write a smallwrapper function. In the solution, this code makes the divide() function return thetwo results using a tuple instead:# int divide(int, int, int *)_divide = _mod.divide_divide.argtypes = (ctypes.c_int, ctypes.c_int, ctypes.POINTER(ctypes.c_int))_divide.restype = ctypes.c_int def divide(x, y):rem = ctypes.c_int()quot = _divide(x,y,rem)return quot, rem.value The avg() function presents a new kind of challenge. The underlying C code expectsto receive a pointer and a length representing an array. However, from the Python side,we must consider the following questions: What is an array? Is it a list? A tuple? Anarray from the array module? A numpy array? Is it all of these? In practice, a Python“array” could take many different forms, and maybe you would like to support multiplepossibilities.The DoubleArrayType class shows how to handle this situation. In this class, a singlemethod from_param() is defined. The role of this method is to take a single parameterand narrow it down to a compatible ctypes object (a pointer to a ctypes.c_double, inthe example). Within from_param(), you are free to do anything that you wish. In thesolution, the typename of the parameter is extracted and used to dispatch to a morespecialized method. For example, if a list is passed, the typename is list and a methodfrom_list() is invoked.For lists and tuples, the from_list() method performs a conversion to a ctypes arrayobject. This looks a little weird, but here is an interactive example of converting a list toa ctypes array: >>> nums = [1, 2, 3] >>> a = (ctypes.c_double * len(nums))(*nums) >>> a <__main__.c_double_Array_3 object at 0x10069cd40> >>> a[0] 1.0 >>> a[1] 2.0 >>> a[2] 3.0 >>> For array objects, the from_array() method extracts the underlying memory pointerand casts it to a ctypes pointer object. For example: >>> import array >>> a = array.array('d',[1,2,3]) >>> a array('d', [1.0, 2.0, 3.0]) >>> ptr_ = a.buffer_info() >>> ptr 4298687200 >>> ctypes.cast(ptr, ctypes.POINTER(ctypes.c_double)) <__main__.LP_c_double object at 0x10069cd40> >>> The from_ndarray() shows comparable conversion code for numpy arrays.By defining the DoubleArrayType class and using it in the type signature of avg(), asshown, the function can accept a variety of different array-like inputs: >>> import sample >>> sample.avg([1,2,3]) 2.0 >>> sample.avg((1,2,3)) 2.0 >>> import array >>> sample.avg(array.array('d',[1,2,3])) 2.0 >>> import numpy >>> sample.avg(numpy.array([1.0,2.0,3.0])) 2.0 >>> The last part of this recipe shows how to work with a simple C structure. For structures,you simply define a class that contains the appropriate fields and types like this: class Point(ctypes.Structure):fields = [(‘x', ctypes.c_double),(‘y', ctypes.c_double)] Once defined, you can use the class in type signatures as well as in code that needs toinstantiate and work with the structures. For example: >>> p1 = sample.Point(1,2) >>> p2 = sample.Point(4,5) >>> p1.x 1.0 >>> p1.y 2.0 >>> sample.distance(p1,p2) 4.242640687119285 >>> A few final comments: ctypes is a useful library to know about if all you’re doing isaccessing a few C functions from Python. However, if you’re trying to access a largelibrary, you might want to look at alternative approaches, such as Swig (described inRecipe 15.9) or Cython (described in Recipe 15.10). The main problem with a large library is that since ctypes isn’t entirely automatic, you’llhave to spend a fair bit of time writing out all of the type signatures, as shown in theexample. Depending on the complexity of the library, you might also have to write alarge number of small wrapper functions and supporting classes. Also, unless you fullyunderstand all of the low-level details of the C interface, including memory managementand error handling, it is often quite easy to make Python catastrophically crash with asegmentation fault, access violation, or some similar error.As an alternative to ctypes, you might also look at CFFI. CFFI provides much of thesame functionality, but uses C syntax and supports more advanced kinds of C code. Asof this writing, CFFI is still a relatively new project, but its use has been growing rapidly.There has even been some discussion of including it in the Python standard library insome future release. Thus, it’s definitely something to keep an eye on.

景凌凯 2020-04-19 21:01:17 0 浏览量 回答数 0

回答

To receive and process arrays in a portable manner, you should write code that uses theBuffer Protocol. Here is an example of a handwritten C extension function that receivesarray data and calls the avg(double *buf, int len) function from this chapter’s in‐troduction: / Call double avg(double [](#), int) */static PyObject *py_avg(PyObject *self, PyObject *args) { PyObject _bufobj;Pybuffer view;double result;/ Get the passed Python object */if (!PyArg_ParseTuple(args, “O”, &bufobj)) { return NULL; } / Attempt to extract buffer information from it [](#)/ if (PyObject_GetBuffer(bufobj, &view,> > PyBUF_ANY_CONTIGUOUS | PyBUF_FORMAT) == -1) { return NULL; } if (view.ndim != 1) {PyErr_SetString(PyExc_TypeError, “Expected a 1-dimensional array”);PyBuffer_Release(&view);return NULL;> } / Check the type of items in the array [](#)/if (strcmp(view.format,”d”) != 0) { PyErr_SetString(PyExc_TypeError, “Expected an array of doubles”);PyBuffer_Release(&view);return NULL; } / Pass the raw buffer and size to the C function [](#)/result = avg(view.buf, view.shape[0]); / Indicate we're done working with the buffer [](#)/PyBuffer_Release(&view);return Py_BuildValue(“d”, result); } Here is an example that shows how this extension function works: >>> import array >>> avg(array.array('d',[1,2,3])) 2.0 >>> import numpy >>> avg(numpy.array([1.0,2.0,3.0])) 2.0 >>> avg([1,2,3]) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: 'list' does not support the buffer interface >>> avg(b'Hello') Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: Expected an array of doubles >>> a = numpy.array([[1.,2.,3.],[4.,5.,6.]]) >>> avg(a[:,2]) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: ndarray is not contiguous >>> sample.avg(a) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: Expected a 1-dimensional array >>> sample.avg(a[0]) 2.0 讨论 Passing array objects to C functions might be one of the most common things you wouldwant to do with a extension function. A large number of Python applications, rangingfrom image processing to scientific computing, are based on high-performance arrayprocessing. By writing code that can accept and operate on arrays, you can write cus‐tomized code that plays nicely with those applications as opposed to having some sortof custom solution that only works with your own code.The key to this code is the PyBuffer_GetBuffer() function. Given an arbitrary Pythonobject, it tries to obtain information about the underlying memory representation. Ifit’s not possible, as is the case with most normal Python objects, it simply raises anexception and returns -1. The special flags passed to PyBuffer_GetBuffer() giveadditional hints about the kind of memory buffer that is requested. For example,PyBUF_ANY_CONTIGUOUS specifies that a contiguous region of memory is required.For arrays, byte strings, and other similar objects, a Py_buffer structure is filled withinformation about the underlying memory. This includes a pointer to the memory, size,itemsize, format, and other details. Here is the definition of this structure: typedef struct bufferinfo {void buf; / Pointer to buffer memory /PyObject *obj; / Python object that is the owner _/Py_ssizet len; / Total size in bytes _/Py_ssizet itemsize; / Size in bytes of a single item /int readonly; / Read-only access flag /int ndim; / Number of dimensions /char *format; / struct code of a single item _/Py_ssizet *shape; / Array containing dimensions _/Py_ssizet *strides; / Array containing strides _/Py_ssizet *suboffsets; / Array containing suboffsets */ } Py_buffer; In this recipe, we are simply concerned with receiving a contiguous array of doubles.To check if items are a double, the format attribute is checked to see if the string is“d”. This is the same code that the struct module uses when encoding binary values.As a general rule, format could be any format string that’s compatible with the structmodule and might include multiple items in the case of arrays containing C structures.Once we have verified the underlying buffer information, we simply pass it to the Cfunction, which treats it as a normal C array. For all practical purposes, it is not con‐cerned with what kind of array it is or what library created it. This is how the functionis able to work with arrays created by the array module or by numpy. Before returning a final result, the underlying buffer view must be released usingPyBuffer_Release(). This step is required to properly manage reference counts ofobjects.Again, this recipe only shows a tiny fragment of code that receives an array. If workingwith arrays, you might run into issues with multidimensional data, strided data, differentdata types, and more that will require study. Make sure you consult the official docu‐mentation to get more details.If you need to write many extensions involving array handling, you may find it easierto implement the code in Cython. See Recipe 15.11.

景凌凯 2020-04-19 21:00:17 0 浏览量 回答数 0

阿里云高校特惠,助力学生创业梦!0元体验,快速入门云计算!

学生动手场景应用,快速了解并掌握云服务器的各种新奇玩法!

回答

目前,Ray支持部分引用计数。(完整的参考计数将很快发布)。简而言之,当传递给远程函数的object_id未序列化时,引用引用的计数方式与引用Python的计数方式相同。这意味着,如果result_transformed是由Python收集的垃圾,result_transformed则应取消固定血浆存储中的,并在将对象清除为LRU时将其清除。(为清楚起见,不会清除具有某些引用计数的固定对象)。 我还假设存在一些奇怪的引用计数,例如循环引用。result_transformed运行该脚本时,我可以验证它是否被驱逐了。因此,我认为result_transformed本身不是问题。可能存在许多问题。就我而言,我发现当我将ipython用作输入(IN)时,它会创建对python对象的引用。(例如,当您看到某个对象的值时,OUT [number]可以引用您的对象)。 In [2]: import psutil ...: import gc ...: import ray ...: from time import sleep ...: import numpy as np ...: @ray.remote ...: def calc_similarity(sims, offset): ...: # Fake some work for 100 ms. ...: sleep(0.10) ...: return True ...: ...: if __name__ == "__main__": ...: # Initialize RAY to use all of the processors. ...: num_cpus = psutil.cpu_count(logical=False) ...: ray.init(num_cpus=num_cpus) ...: ...: num_docs = 1000000 ...: num_dimensions = 300 ...: chunk_size = 128 ...: sim_pct = 0.82 ...: ...: # Initialize the array ...: index = np.random.random((num_docs, num_dimensions)).astype(dtype=np.float32) ...: index_array = np.arange(num_docs).reshape(1, num_docs) ...: index_array_id = ray.put(index_array) ...: ...: calc_results = [] ...: i = 0 ...: for count, start_doc_no in enumerate(range(0, num_docs, chunk_size)): ...: i += 1 ...: size = min( chunk_size, num_docs - (start_doc_no) + 1 ) ...: # Get the query vector out of the index. ...: query_vector = index[start_doc_no:start_doc_no+size] ...: # Calculate the matrix multiplication. ...: result_transformed = np.matmul(index, query_vector.T).T ...: # Serialize the result matrix out for each client. ...: result_id = ray.put(result_transformed) ...: if i == 1: ...: # The first result_id binary number should be stored in result_id_special ...: # In this way, we can verify if this object id is evicted after filling up our ...: # plasma store by some random numpy array ...: # If this object id is not evicted, that means it is pinned, meaning if is ...: # not properly reference counted. ...: first_object_id = result_id.binary() ...: # Simulate multi-threading extracting the results of a cosine similarity calculation ...: for offset in range(chunk_size): ...: calc_results.append(calc_similarity.remote(sims=result_id, offset=offset )) ...: # , index_array=index_array_id)) ...: res = ray.get(calc_results) ...: calc_results.clear() ...: print('ref count to result_id {}'.format(len(gc.get_referrers(result_id)))) ...: print('Total number of ref counts in a ray cluster. {}'.format(ray.worker.global_worker.core_worker.get_all_reference_counts())) ...: if i == 5: ...: break ...: # It should contain the object id of the ...: print('first object id: {}'.format(first_object_id)) ...: print('fill up plasma store by big numpy arrays. This should evict the first_object_id from the plasma store.') ...: print('because if the data_transformed is garbage collected properly, it should be unpinned from plasma store') ...: print('and when plasma store is filled by numpy array, first_object_id should be evicted.') ...: for _ in range(40): ...: import numpy as np ...: ray.put(np.zeros(500 * 1024 * 1024, dtype=np.uint8)) ...: print('total ref count from a ray cluster after eviction: {}'.format(ray.worker.global_worker.core_worker.get_all_reference_counts())) ...: # this should fail as first_object_id is already evicted ...: print(ray.get(ray.ObjectID(first_object_id))) [ray] Forcing OMP_NUM_THREADS=1 to avoid performance degradation with many workers (issue #6998). You can override this by explicitly setting OMP_NUM_THREADS. 2020-02-12 00:10:11,932 INFO resource_spec.py:212 -- Starting Ray with 4.35 GiB memory available for workers and up to 2.19 GiB for objects. You can adjust these settings with ray.init(memory=<bytes>, object_store_memory=<bytes>). 2020-02-12 00:10:12,273 INFO services.py:1080 -- View the Ray dashboard at localhost:8265 2020-02-12 00:10:18,522 WARNING worker.py:289 -- OMP_NUM_THREADS=1 is set, this may slow down ray.put() for large objects (issue #6998). ref count to result_id 1 Total number of ref counts in a ray cluster. {ObjectID(ffffffffffffffffffffffff0100008002000000): {'local': 1, 'submitted': 0}, ObjectID(ffffffffffffffffffffffff0100008001000000): {'local': 1, 'submitted': 0}} ref count to result_id 1 Total number of ref counts in a ray cluster. {ObjectID(ffffffffffffffffffffffff0100008003000000): {'local': 1, 'submitted': 0}, ObjectID(ffffffffffffffffffffffff0100008001000000): {'local': 1, 'submitted': 0}} ref count to result_id 1 Total number of ref counts in a ray cluster. {ObjectID(ffffffffffffffffffffffff0100008001000000): {'local': 1, 'submitted': 0}, ObjectID(ffffffffffffffffffffffff0100008004000000): {'local': 1, 'submitted': 0}} ref count to result_id 1 Total number of ref counts in a ray cluster. {ObjectID(ffffffffffffffffffffffff0100008001000000): {'local': 1, 'submitted': 0}, ObjectID(ffffffffffffffffffffffff0100008005000000): {'local': 1, 'submitted': 0}} ref count to result_id 1 Total number of ref counts in a ray cluster. {ObjectID(ffffffffffffffffffffffff0100008006000000): {'local': 1, 'submitted': 0}, ObjectID(ffffffffffffffffffffffff0100008001000000): {'local': 1, 'submitted': 0}} first object id: b'\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x01\x00\x00\x80\x02\x00\x00\x00' fill up plasma store by big numpy arrays. This should evict the first_object_id from the plasma store. because if the data_transformed is garbage collected properly, it should be unpinned from plasma store and when plasma store is filled by numpy array, first_object_id should be evicted. total ref count from a ray cluster after eviction: {ObjectID(ffffffffffffffffffffffff0100008006000000): {'local': 1, 'submitted': 0}, ObjectID(ffffffffffffffffffffffff0100008001000000): {'local': 1, 'submitted': 0}} 2020-02-12 00:10:57,108 WARNING worker.py:1515 -- Local object store memory usage: num clients with quota: 0 quota map size: 0 pinned quota map size: 0 allocated bytes: 2092865189 allocation limit: 2347285708 pinned bytes: 520000477 (global lru) capacity: 2347285708 (global lru) used: 67.0078% (global lru) num objects: 4 (global lru) num evictions: 41 (global lru) bytes evicted: 21446665725 2020-02-12 00:10:57,112 WARNING worker.py:1072 -- The task with ID ffffffffffffffffffffffff0100 is a driver task and so the object created by ray.put could not be reconstructed. --------------------------------------------------------------------------- UnreconstructableError Traceback (most recent call last) <ipython-input-1-184e5836123c> in <module> 63 print('total ref count from a ray cluster after eviction: {}'.format(ray.worker.global_worker.core_worker.get_all_reference_counts())) 64 # this should fail as first_object_id is already evicted ---> 65 print(ray.get(ray.ObjectID(first_object_id))) 66 ~/work/ray/python/ray/worker.py in get(object_ids, timeout) 1517 raise value.as_instanceof_cause() 1518 else: -> 1519 raise value 1520 1521 # Run post processors. UnreconstructableError: Object ffffffffffffffffffffffff0100008002000000 is lost (either LRU evicted or deleted by user) and cannot be reconstructed. Try increasing the object store memory available with ray.init(object_store_memory=<bytes>) or setting object store limits with ray.remote(object_store_memory=<bytes>). See also: https://ray.readthedocs.io/en/latest/memory-management.html

祖安文状元 2020-02-22 10:27:48 0 浏览量 回答数 0

问题

修改了php.ini各种重启依旧无效:配置报错 

kun坤 2020-06-04 10:23:30 4 浏览量 回答数 1

问题

PHP MySQL Google Chart JSON-完整示例

保持可爱mmm 2020-05-11 10:58:55 0 浏览量 回答数 1

回答

要访问array或object您如何使用两个不同的运算符。 数组 要访问数组元素,您必须使用,[]或者您不会看到太多,但也可以使用is {}。 echo $array[0]; echo $array{0}; //Both are equivalent and interchangeable 声明数组与访问数组元素之间的区别 定义数组和访问数组元素是两件事。所以不要把它们混在一起。 要定义一个数组,可以使用array()或对于PHP> = 5.4 []并分配/设置一个数组/元素。当您使用[]或{}如上所述访问数组元素时,将获得与设置元素相反的数组元素的值。 // 声明一个数组 $ arrayA = array( / 这里有一些东西 / ) ; $ arrayB = [ / 这里有一些东西 / ];//仅适用于PHP> = 5.4 // 访问数组元素 echo $ array [ 0 ] ; 回声$ array { 0 } ; 访问数组元素 要访问数组中的特定元素,可以使用内部的任何表达式,[]或者{}将其求值为要访问的键: $ array [ (任何表达式) ] 因此,请注意使用什么表达式作为键,以及如何通过PHP对其进行解释: echo $ array [ 0 ]; //键是一个整数;它访问0的元素 echo $ array [ “ 0” ]; //键是一个字符串;它访问0的元素 echo $ array [ “ string” ]; //键是一个字符串;它使用键“ string”访问元素 echo $ array [ CONSTANT ]; //键是一个常量,它被替换为对应的值 echo $ array [ cOnStAnT ]; //键也是常量而不是字符串 echo $ array [ $ anyVariable ] //键是一个变量,它被替换为'$ anyVariable'中的值 echo $ array [ functionXY() ]; //键将是函数 的返回值 访问多维数组 如果彼此之间有多个数组,则只需一个多维数组。要访问子数组中的数组元素,只需使用multiple即可[]。 echo $array["firstSubArray"]["SecondSubArray"]["ElementFromTheSecondSubArray"] // ├─────────────┘ ├──────────────┘ ├────────────────────────────┘ // │ │ └── 3rd Array dimension; // │ └──────────────────── 2d Array dimension; // └───────────────────────────────────── 1st Array dimension; 对象 要访问对象属性,必须使用->。 echo $ object- >属性; 如果在另一个对象中有一个对象,则只需使用多个->即可获得对象属性。 echo $objectA->objectB->property; 注意: 另外,如果您使用的属性名称无效,也必须小心!因此,要查看所有问题,您可能会遇到一个无效的属性名称,请参阅此问题/答案。如果您在属性名称的开头有数字,则尤其要注意这一点。 您只能从班级外部访问具有公共可见性的属性。否则(私有或受保护的),您需要一个方法或反射,您可以使用该方法或反射来获取属性的值。 数组与对象 现在,如果您将数组和对象彼此混合在一起,则只需查看是否现在访问数组元素或对象属性并为其使用相应的运算符即可。 //宾语 echo $ object-> anotherObject-> propertyArray [“ elementOneWithAnObject”]-> property; //├────┘├──────────┘├─────────────├─────────── ───────┘├──────┘ //││││└──属性; //│││└────────────────────────数组元素(对象);使用->访问属性“ property” //││└──────────────────────数组(财产);使用[]访问数组元素'elementOneWithAnObject' //│└──────────────────────────── ────────────属性(对象);使用->访问属性'propertyArray' //└────────────────────────────── ───────────────────对象;使用->访问属性“ anotherObject” //数组 echo $ array [“ arrayElement”] [“ anotherElement”]-> object-> property [“ element”]; //├───┘├──────────┘├────────────┘├────┘├────── ├├───────┘ //│││││└──数组元素; //││││└────────────属性(数组);使用[]访问数组元素'element' //│││└────────────────────属性(对象);使用->访问属性“ property” //││└────────────────数组元素(对象);使用->访问属性“对象” //│└──────────────────────────── ────────数组元素(array); 使用[]访问数组元素'anotherElement' //└────────────────────────────── ─────────────数组 使用[]访问数组元素'arrayElement' 我希望这给您一个大概的想法,当它们相互嵌套时如何访问数组和对象。 注意: 是否调用数组或对象取决于变量的最外部。所以[new StdClass]是一个阵列,即使它已(嵌套)对象内的,并$object->property = array();是一个对象,即使它已(嵌套)阵列内。 并且,如果不确定是否有对象或数组,请使用gettype()。 如果有人使用您以外的其他编码样式,请不要感到困惑: //Both methods/styles work and access the same data echo $object->anotherObject->propertyArray["elementOneWithAnObject"]->property; echo $object-> anotherObject ->propertyArray ["elementOneWithAnObject"]-> property; //Both methods/styles work and access the same data echo $array["arrayElement"]["anotherElement"]->object->property["element"]; echo $array["arrayElement"] ["anotherElement"]-> object ->property["element"]; 数组,对象和循环 如果您不仅要访问单个元素,还可以遍历嵌套的数组/对象并遍历特定维的值。 为此,您只需要访问要循环的维度,然后就可以循环浏览该维度的所有值。 作为示例,我们采用一个数组,但它也可以是一个对象: Array ( [data] => Array ( [0] => stdClass Object ( [propertyXY] => 1 ) [1] => stdClass Object ( [propertyXY] => 2 ) [2] => stdClass Object ( [propertyXY] => 3 ) ) ) 如果在第一个维度上循环,则将从第一个维度获取所有值: foreach($ array as $ key => $ value) 这意味着在第一维中,您只有一个带有key($key)data和value($value)的元素: Array ( //Key: array [0] => stdClass Object ( [propertyXY] => 1 ) [1] => stdClass Object ( [propertyXY] => 2 ) [2] => stdClass Object ( [propertyXY] => 3 ) ) 如果在第二维上循环,则将从第二维获取所有值: foreach($ array [“ data”] as $ key => $ value) 在这里意味着在第二个维度你有3个元素与键($key)0,1,2和值($value): stdClass Object ( //Key: 0 [propertyXY] => 1 ) stdClass Object ( //Key: 1 [propertyXY] => 2 ) stdClass Object ( //Key: 2 [propertyXY] => 3 ) 这样,您就可以遍历任何维,无论它是数组还是对象。 分析var_dump()/ print_r()/ var_export()输出 所有这三个调试功能都输出相同的数据,只是以另一种格式或带有一些元数据(例如,类型,大小)。因此,在这里我想展示如何读取这些函数的输出,以了解/了解如何从数组/对象访问某些数据。 输入数组: $array = [ "key" => (object) [ "property" => [1,2,3] ] ]; var_dump() 输出: array(1) { ["key"]=> object(stdClass)#1 (1) { ["property"]=> array(3) { [0]=> int(1) [1]=> int(2) [2]=> int(3) } } } print_r() 输出: Array ( [key] => stdClass Object ( [property] => Array ( [0] => 1 [1] => 2 [2] => 3 ) ) ) var_export() 输出: array ( 'key' => stdClass::__set_state(array( 'property' => array ( 0 => 1, 1 => 2, 2 => 3, ), )), ) 因此,您可以看到所有输出都非常相似。而且,如果现在要访问值2,则可以从值本身开始,您要访问该值,然后逐步到达“左上角”。 1.我们首先看到,值2在键为1的数组中 array(3){ // var_dump() [0] => 整数(1) [1] => 整数(2) [2] => 整数(3) } 数组 // print_r() ( [0] => 1 [1] => 2 [2] => 3 ) 数组( // var_export() 0 => 1 1 => 2 2 => 3, ), 这意味着我们必须使用[]/ {}来访问值2 [1],因为该值具有键/索引1。 2.接下来,我们看到将数组分配给具有对象名称属性的属性 对象(stdClass)#1(1){ // var_dump() [“ property”] => / 此处数组 / } stdClass 对象 // print_r() ( [property] => / 此处为数组 / ) stdClass :: __ set_state ( array(// var_export() 'property' => / 此处数组 / )) 这意味着我们必须使用->访问对象的属性,例如->property。 所以直到现在我们知道,我们必须使用->property[1]。 3.最后,我们看到最外面的是一个数组 array(1){ // var_dump() [“ key”] => / 对象和数组在这里 / } 数组 // print_r() ( [key] => / 对象和数组在这里 / ) 数组( // var_export() 'key' => / 对象和数组在这里 / ) 如我们所知,我们必须使用来访问数组元素[],我们在这里看到我们必须使用["key"]来访问对象。现在,我们可以将所有这些部分放在一起并编写: echo $array["key"]->property[1]; 输出将是: 2 不要让PHP欺骗您! 您需要了解一些事情,这样您就不必花费大量时间来寻找它们。 “隐藏”字符 有时,您的按键中包含字符,这些字符在浏览器的第一次外观中不会出现。然后您要问自己,为什么无法访问该元素。这些字符可以是:标签(\t),新线(\n),空格或HTML标签(例如, )等。 作为示例,如果您查看的输出,print_r()则会看到: Array ( [key] => HERE ) 然后,您尝试通过以下方式访问元素: echo $arr["key"]; 但是您会收到通知: 注意:未定义的索引:键 这很好地表明必须有一些隐藏的字符,因为即使键看起来很正确,也无法访问该元素。 这里的技巧是使用var_dump()+查看源代码!(备选:highlight_string(print_r($variable, TRUE));) 突然之间,您可能会看到以下内容: array(1) { [" key"]=> string(4) "HERE" } 现在您将看到,您的键中带有一个html标记+一个换行符,这是您最初没有看到的,因为print_r()浏览器没有显示它。 所以现在,如果您尝试执行以下操作: echo $arr["\nkey"]; 您将获得所需的输出: HERE 永远不要相信XML 的输出,print_r()或者var_dump()如果您看XML 您可能将XML文件或字符串加载到对象中,例如 现在,如果您使用var_dump()或,print_r()您将看到: SimpleXMLElement Object ( [item] => SimpleXMLElement Object ( [title] => test ) ) 如您所见,您看不到标题的属性。因此,正如我说永远不要相信的输出var_dump()或print_r()当你有一个XML对象。始终用于asXML()查看完整的XML文件/字符串。 因此,只需使用下面显示的方法之一: echo $xml->asXML(); //And look into the source code highlight_string($xml->asXML()); header ("Content-Type:text/xml"); echo $xml->asXML(); 然后您将获得输出:

保持可爱mmm 2020-02-09 14:02:47 0 浏览量 回答数 0

问题

E-MapReducePig 作业配置是什么?

nicenelly 2019-12-01 21:17:11 1331 浏览量 回答数 0

问题

E-MapReducePig 作业配置是什么?

nicenelly 2019-12-01 21:21:04 870 浏览量 回答数 0

问题

E-MapReduce HadoopPig 开发手册是什么?

nicenelly 2019-12-01 21:22:44 869 浏览量 回答数 0

问题

如何获取pdf文档iText 7的页面大小

montos 2020-03-23 14:59:12 1 浏览量 回答数 1

问题

用PHP将数组插入MySQL数据库

保持可爱mmm 2020-05-10 22:19:08 0 浏览量 回答数 1

回答

简介 HPL(the High-Performance Linpack Benchmark)是国际上最流行的用于测试高性能计算机系统浮点性能的benchmark。通过对高性能计算机采用高斯消元法求解一元N次稠密线性代数方程组的测试,评价高性能计算机的浮点性能。浮点计算峰值是指计算机每秒钟能完成的浮点计算最大次数。包括理论浮点峰值和实测浮点峰值。理论浮点峰值是该计算机理论上能达到的每秒钟能完成浮点计算最大次数,它主要是由CPU的主频决定的。 理论浮点峰值 = CPU主频 × CPU每个时钟周期执行浮点运算的次数 × 系统中CPU数 准备工作 若您尚未拥有E-HPC集群,请先创建E-HPC集群 运行以下示例需要在创建集群时或者软件管理界面上选择安装linpack软件包和intel-mpi通信库。勾选linpack勾选intel mpi 输入参数说明 输入文件HPL.dat包含了HPL的运行参数,下图是在单台scch5实例上运行HPL的推荐配置。 HPLinpack benchmark input file Innovative Computing Laboratory, University of Tennessee HPL.out output file name (if any) 6 device out (6=stdout,7=stderr,file) 1 # of problems sizes (N) 143360 256000 1000 Ns 1 # of NBs 384 192 256 NBs 1 PMAP process mapping (0=Row-,1=Column-major) 1 # of process grids (P x Q) 1 2 Ps 1 2 Qs 16.0 threshold 1 # of panel fact 2 1 0 PFACTs (0=left, 1=Crout, 2=Right) 1 # of recursive stopping criterium 2 NBMINs (>= 1) 1 # of panels in recursion 2 NDIVs 1 # of recursive panel fact. 1 0 2 RFACTs (0=left, 1=Crout, 2=Right) 1 # of broadcast 0 BCASTs (0=1rg,1=1rM,2=2rg,3=2rM,4=Lng,5=LnM) 1 # of lookahead depth 0 DEPTHs (>=0) 0 SWAP (0=bin-exch,1=long,2=mix) 1 swapping threshold 1 L1 in (0=transposed,1=no-transposed) form 1 U in (0=transposed,1=no-transposed) form 0 Equilibration (0=no,1=yes) 8 memory alignment in double (> 0) 测试过程中需要根据节点硬件配置而做出调整的运行参数主要有: 第5、6行:代表求解的矩阵数量与规模。矩阵规模N越大,有效计算所占的比例也越大,系统浮点处理性能也就越高;但与此同时,矩阵规模N的增加会导致内存消耗量的增加,一旦系统实际内存空间不足,使用缓存、性能会大幅度降低。矩阵占用系统总内存的80%左右为最佳,即N x N x 8 = 系统总内存 x 80% (其中总内存换算以字节为单位)。 第7、8行:代表求解矩阵过程中矩阵分块的大小。分块大小对性能有很大的影响,NB的选择和软硬件许多因素密切相关。NB值的选择主要是通过实际测试得出最优值,但还是有一些规律可循:NB不能太大或太小,一般在384以下;NB × 8一定是Cache line的倍数等。例如,L2 cache为1024K, NB就设置为192。另外,NB大小的选择还跟通信方式、矩阵规模、网络、处理器速度等有关系。一般通过单节点或单CPU测试可以得到几个较好的NB值,但当系统规模增加、问题规模变大,有些NB取值所得性能会下降。所以最好在小规模测试时选择3个左右性能不错的NB,再通过大规模测试检验这些选择。 第10~12行:代表二维处理器网格(P × Q)。P × Q = 系统CPU数 = 进程数。一般来说一个进程对于一个CPU可以得到最佳性能。对于Intel Xeon来说,关闭超线程可以提高HPL性能。P≤Q;一般来说,P的值尽量取得小一点,因为列向通信量(通信次数和通信数据量)要远大于横向通信。P = 2n,即P最好选择2的幂。HPL中,L分解的列向通信采用二元交换法(Binary Exchange),当列向处理器个数P为2的幂时,性能最优。例如,当系统进程数为4的时候,P × Q选择为1 × 4的效果要比选择2 × 2好一些。 在集群测试中,P × Q = 系统CPU总核数。 运行HPL测试 E-HPC控制台创建HPL.dat输入文件 返回E-HPC管理控制台,点选左侧栏的“作业”标签,进入作业管理界面。依次选择“创建作业”->“新建文件”->“使用文件模板”->“HPL.dat”,根据节点硬件配置调整HPL输入参数,得到HPL输入文件如下。 HPL.dat E-HPC控制台创建HPL.pbs作业脚本 在作业管理界面中,依次选择“创建作业”->“新建文件”->“使用文件模板”->“pbs demo”,对pbs demo脚本进行修改,得到HPL作业脚本HPL.pbs如下。 #!/bin/sh #PBS -j oe export MODULEPATH=/opt/ehpcmodulefiles/ module load linpack/2018 module load intel-mpi/2018 echo "run at the beginning" mpirun -n 1 -host /opt/linpack/2018/xhpl_intel64_static > hpl-ouput #测试单节点的浮点性能 mpirun -n -ppn 1 -host ,..., /opt/linpack/2018/xhpl_intel64_static > hpl-ouput #测试多节点的浮点性能 E-HPC控制台提交HPL测试作业 确定下图左侧作业基本参数后,点击右上角“确认”提交作业。作业个性化配置、作业导入、作业导出以及作业状态查看,请参见作业管理。 作业配置 E-HPC控制台查询作业状态 点击作业列表中HPL作业右侧的 “详情” 按钮,查看作业详细信息。 作业详细信息 E-HPC控制台查看结果文件 返回E-HPC管理控制台,点选集群右侧“更多”选项,选择“执行命令”,进入集群命令运行界面。 执行命令 在集群命令运行界面点击“批量执行”,选择集群登录/管控节点执行命令,查看HPL作业结果文件。 查看结果 从结果文件中获取测得的HPL浮点运算效率数据,格式如下。 T/V N NB P Q Time Gflops WC00C2R2 143360 384 1 1 XXXX XXXXXXX 简介 IMB (Intel MPI Benchmarks) 用于评估HPC集群在不同消息粒度下节点间点对点、全局通信的效率。 准备工作 若您尚未拥有E-HPC集群,请先创建E-HPC集群 运行以下示例需要在创建集群时或者软件管理界面上选择安装intel-mpi-benchmarks软件包和intel-mpi通信库 勾选IMBintel-mpi IMB测试方法说明 $ /opt/intel-mpi-benchmarks/2019/IMB-MPI1 -h #查看IMB支持的通信模式及参数说明 $ cd /home/ /<work_dir> #非root用户下执行 $ /opt/intel/impi/2018.3.222/bin64/mpirun -genv I_MPI_DEBUG 5 -np 2 -ppn 1 -host , /opt/intel-mpi-benchmarks/2019/IMB-MPI1 pingpong #测试两节点间pingpong通信模式效率,获取通信延迟和带宽 $ /opt/intel/impi/2018.3.222/bin64/mpirun -genv I_MPI_DEBUG 5 -np <N*2> -ppn 2 -host ,..., /opt/intel-mpi-benchmarks/2019/IMB-MPI1 -npmin 2 -msglog 19:21 allreduce #测试N节点间allreduce通信模式效率,每个节点开启两个进程,获取不同消息粒度下的通信时间 $ /opt/intel/impi/2018.3.222/bin64/mpirun -genv I_MPI_DEBUG 5 -np -ppn 1 -host ,..., /opt/intel-mpi-benchmarks/2019/IMB-MPI1 -npmin 1 -msglog 15:17 alltoall #测试N节点间alltoall通信模式效率,每个节点开启一个进程,获取不同消息粒度下的通信时间 ############关键参数说明############# -genv I_MPI_DEBUG 打印mpi debug信息 -np 指定mpi总进程数 -ppn 指定每个节点的进程数 -host 指定任务节点列表 -npmin 指定至少运行的进程数 -msglog 指定消息片粒度范围 运行IMB测试 E-HPC控制台创建IMB.pbs作业脚本 在作业管理界面中,依次选择“创建作业”->“新建文件”->“使用文件模板”->“pbs demo”,对pbs demo脚本进行修改,得到IMB作业脚本IMB.pbs如下。 #!/bin/sh #PBS -j oe #PBS -l select=2:ncpus= :mpiprocs=1 #N为节点CPU核数,实际测试中根据节点配置进行设置 export MODULEPATH=/opt/ehpcmodulefiles/ module load intel-mpi/2018 module load intel-mpi-benchmarks/2019 echo "run at the beginning" /opt/intel/impi/2018.3.222/bin64/mpirun -genv I_MPI_DEBUG 5 -np 2 -ppn 1 -host compute0,compute1 /opt/intel-mpi-benchmarks/2019/IMB-MPI1 pingpong > IMB-pingpong E-HPC控制台提交IMB测试作业 确定下图左侧作业基本参数后,点击右上角“确认”提交作业。作业个性化配置、作业导入、作业导出以及作业状态查看,请参见作业管理。 作业提交 E-HPC控制台查看结果文件 从E-HPC管理控制台,点选集群右侧“更多”选项,选择“执行命令”,进入集群命令运行界面。点击“批量执行”,选择集群登录/管控节点执行命令,查看IMB作业结果文件。 作业结果 简介 STREAM测试是内存测试中业界公认的内存带宽性能测试基准工具,是衡量服务器内存性能指标的通用工具。STREAM具有良好的空间局部性,是对 TLB 友好、Cache友好的一款测试,支持Copy、Scale、Add、Triad四种操作。 准备工作 若您尚未拥有E-HPC集群,请先创建E-HPC集群 运行以下示例需要在创建集群时或者软件管理界面上选择安装STREAM软件包 勾选stream 运行STREAM测试 E-HPC控制台编译STREAM 为了避免数据Cache重用对测试结果准确度产生较大影响,需确保STREAM开辟的数组大小远大于L3 Cache的容量且小于内存的容量。因此在实际测试中要根据测试节点配置对STREAM进行重新编译。由E-HPC管理控制台进入集群命令运行界面,登录节点执行如下操作。 编译stream $ cd /opt/stream/2018/; gcc stream.c -O3 -fopenmp -DSTREAM_ARRAY_SIZE=102410241024 -DNTIMES=20 -mcmodel=medium -o stream.1g.20 #-DSTREAM_ARRAY_SIZE用于指定STREAM一次搬运的数据量,-DTIMES用于指定迭代次数 E-HPC控制台创建STREAM.pbs作业脚本 在作业管理界面中,依次选择“创建作业”->“新建文件”->“使用文件模板”->“pbs demo”,对pbs demo脚本进行修改,得到STREAM作业脚本STREAM.pbs如下。 #!/bin/sh #PBS -j oe #PBS -l select=1:ncpus= #N为节点CPU核数,实际测试中根据节点配置进行设置 export MODULEPATH=/opt/ehpcmodulefiles/ module load stream/2018 echo "run at the beginning" OMP_NUM_THREADS=1 /opt/stream/stream.1g.20 > stream-1-thread.log OMP_NUM_THREADS=2 /opt/stream/stream.1g.20 > stream-2-thread.log OMP_NUM_THREADS=4 /opt/stream/stream.1g.20 > stream-4-thread.log OMP_NUM_THREADS=8 /opt/stream/stream.1g.20 > stream-8-thread.log ... OMP_NUM_THREADS= /opt/stream/stream.1g.20 > stream- -thread.log E-HPC控制台提交STREAM测试作业 确定下图左侧作业基本参数后,点击右上角“确认”提交作业。作业个性化配置、作业导入、作业导出以及作业状态查看,请参见作业管理。 提交作业 E-HPC控制台查看结果文件 从E-HPC管理控制台,点选集群右侧“更多”选项,选择“执行命令”,进入集群命令运行界面。点击“批量执行”,选择集群登录/管控节点执行命令,查看STREAM作业结果文件。 stream结果 简介 STREAM测试是内存测试中业界公认的内存带宽性能测试基准工具,是衡量服务器内存性能指标的通用工具。STREAM具有良好的空间局部性,是对 TLB 友好、Cache友好的一款测试,支持Copy、Scale、Add、Triad四种操作。 准备工作 若您尚未拥有E-HPC集群,请先创建E-HPC集群 运行以下示例需要在创建集群时或者软件管理界面上选择安装STREAM软件包 勾选stream 运行STREAM测试 E-HPC控制台编译STREAM 为了避免数据Cache重用对测试结果准确度产生较大影响,需确保STREAM开辟的数组大小远大于L3 Cache的容量且小于内存的容量。因此在实际测试中要根据测试节点配置对STREAM进行重新编译。由E-HPC管理控制台进入集群命令运行界面,登录节点执行如下操作。 编译stream $ cd /opt/stream/2018/; gcc stream.c -O3 -fopenmp -DSTREAM_ARRAY_SIZE=102410241024 -DNTIMES=20 -mcmodel=medium -o stream.1g.20 #-DSTREAM_ARRAY_SIZE用于指定STREAM一次搬运的数据量,-DTIMES用于指定迭代次数 E-HPC控制台创建STREAM.pbs作业脚本 在作业管理界面中,依次选择“创建作业”->“新建文件”->“使用文件模板”->“pbs demo”,对pbs demo脚本进行修改,得到STREAM作业脚本STREAM.pbs如下。 #!/bin/sh #PBS -j oe #PBS -l select=1:ncpus= #N为节点CPU核数,实际测试中根据节点配置进行设置 export MODULEPATH=/opt/ehpcmodulefiles/ module load stream/2018 echo "run at the beginning" OMP_NUM_THREADS=1 /opt/stream/stream.1g.20 > stream-1-thread.log OMP_NUM_THREADS=2 /opt/stream/stream.1g.20 > stream-2-thread.log OMP_NUM_THREADS=4 /opt/stream/stream.1g.20 > stream-4-thread.log OMP_NUM_THREADS=8 /opt/stream/stream.1g.20 > stream-8-thread.log ... OMP_NUM_THREADS= /opt/stream/stream.1g.20 > stream- -thread.log E-HPC控制台提交STREAM测试作业 确定下图左侧作业基本参数后,点击右上角“确认”提交作业。作业个性化配置、作业导入、作业导出以及作业状态查看,请参见作业管理。 提交作业 E-HPC控制台查看结果文件 从E-HPC管理控制台,点选集群右侧“更多”选项,选择“执行命令”,进入集群命令运行界面。点击“批量执行”,选择集群登录/管控节点执行命令,查看STREAM作业结果文件。 stream结果

1934890530796658 2020-03-23 18:05:57 0 浏览量 回答数 0

问题

如何从foreach循环内的数组中删除对象?

保持可爱mmm 2020-02-07 01:02:25 1 浏览量 回答数 1

问题

通过`in-code variable inspection`调试scala中的过滤器操作[重复]

社区小助手 2019-12-01 19:29:07 378 浏览量 回答数 1

回答

这两天净回答跟 DOM 有关的问题了 ... 也不知道是为什么 ... 额 ... 以上是题外话 ... 你这个问题 ... 如果你都能写到这里了 ... 再继续一步应该不是什么难事 ... 编程除了需要严谨 ... 有时候在遇到不知道的问题时也需要天马行空的想象力和不怕碰壁的尝试 ... <?php /* just copied six lines below ... */ $Browser = new COM('InternetExplorer.Application'); $Browserhandle = $Browser->HWND; $Browser->Visible = true; $f1_url = "http://**.com/def.php"; $Browser->Navigate($f1_url); sleep(5); /* $allforms is NOT an array ... it is an iterator ... */ $allforms = $Browser->Document->getElementsByTagName( 'form' ); /* current() is not implemented ... so we have to run a loop ... */ foreach( $allforms as $theform ) { /* i also copied these six lines and did some text replace work ... */ $theform->id->focus(); $theform->id->value = "username"; $theform->pwd->focus(); $theform->pwd->value = "password"; $theform->action->focus(); $theform->action->click(); } /* bye browser ... have a nice day ... */ $Browser->Quit(); 事实上你可以处理网页上的所有元素 ... 都是这一个道理 ...

杨冬芳 2019-12-02 02:46:58 0 浏览量 回答数 0

回答

逻辑回归 逻辑回归实际上是一种分类算法。我怀疑它这样命名是因为它与线性回归在学习方法上很相似,但是成本和梯度函数表述不同。特别是,逻辑回归使用了一个sigmoid或“logit”激活函数,而不是线性回归的连续输出。 首先导入和检查我们将要处理的数据集。 import numpy as np import pandas as pd import matplotlib.pyplot as plt %matplotlib inline import os path = os.getcwd() + '\data\ex2data1.txt' data = pd.read_csv(path, header=None, names=['Exam 1', 'Exam 2', 'Admitted']) data.head() 在数据中有两个连续的自变量——“Exam 1”和“Exam 2”。我们的预测目标是“Admitted”的标签。值1表示学生被录取,0表示学生没有被录取。我们看有两科成绩的散点图,并使用颜色编码来表达例子是positive或者negative。 positive = data[data['Admitted'].isin([1])] negative = data[data['Admitted'].isin([0])] fig, ax = plt.subplots(figsize=(12,8)) ax.scatter(positive['Exam 1'], positive['Exam 2'], s=50, c='b', marker='o', label='Admitted') ax.scatter(negative['Exam 1'], negative['Exam 2'], s=50, c='r', marker='x', label='Not Admitted') ax.legend() ax.set_xlabel('Exam 1 Score') ax.set_ylabel('Exam 2 Score') 从这个图中我们可以看到,有一个近似线性的决策边界。它有一点弯曲,所以我们不能使用直线将所有的例子正确地分类,但我们能够很接近。现在我们需要实施逻辑回归,这样我们就可以训练一个模型来找到最优决策边界,并做出分类预测。首先需要实现sigmoid函数。 def sigmoid(z): return 1 / (1 + np.exp(-z)) 这个函数是逻辑回归输出的“激活”函数。它将连续输入转换为0到1之间的值。这个值可以被解释为分类概率,或者输入的例子应该被积极分类的可能性。利用带有界限值的概率,我们可以得到一个离散标签预测。它有助于可视化函数的输出,以了解它真正在做什么。 nums = np.arange(-10, 10, step=1) fig, ax = plt.subplots(figsize=(12,8)) ax.plot(nums, sigmoid(nums), 'r') 我们的下一步是写成本函数。成本函数在给定一组模型参数的训练数据上评估模型的性能。这是逻辑回归的成本函数。 def cost(theta, X, y): theta = np.matrix(theta) X = np.matrix(X) y = np.matrix(y) first = np.multiply(-y, np.log(sigmoid(X * theta.T))) second = np.multiply((1 - y), np.log(1 - sigmoid(X * theta.T))) return np.sum(first - second) / (len(X)) 注意,我们将输出减少到单个标量值,该值是“误差”之和,是模型分配的类概率与示例的真实标签之间差别的量化函数。该实现完全是向量化的——它在语句(sigmoid(X * theta.T))中计算模型对整个数据集的预测。 测试成本函数以确保它在运行,首先需要做一些设置。 # add a ones column - this makes the matrix multiplication work out easier data.insert(0, 'Ones', 1) # set X (training data) and y (target variable) cols = data.shape[1] X = data.iloc[:,0:cols-1] y = data.iloc[:,cols-1:cols] # convert to numpy arrays and initalize the parameter array theta X = np.array(X.values) y = np.array(y.values) theta = np.zeros(3) 检查数据结构的形状,以确保它们的值是合理的。这种技术在实现矩阵乘法时非常有用 X.shape, theta.shape, y.shape ((100L, 3L), (3L,), (100L, 1L)) 现在计算初始解的成本,将模型参数“theta”设置为零。 cost(theta, X, y) 0.69314718055994529 我们已经有了工作成本函数,下一步是编写一个函数,用来计算模型参数的梯度,以找出改变参数来提高训练数据模型的方法。在梯度下降的情况下,我们不只是在参数值周围随机地jigger,看看什么效果最好。并且在每次迭代训练中,我们通过保证将其移动到减少训练误差(即“成本”)的方向来更新参数。我们可以这样做是因为成本函数是可微分的。这是函数。 def gradient(theta, X, y): theta = np.matrix(theta) X = np.matrix(X) y = np.matrix(y) parameters = int(theta.ravel().shape[1]) grad = np.zeros(parameters) error = sigmoid(X * theta.T) - y for i in range(parameters): term = np.multiply(error, X[:,i]) grad[i] = np.sum(term) / len(X) return grad 我们并没有在这个函数中执行梯度下降——我们只计算一个梯度步骤。在练习中,使用“fminunc”的Octave函数优化给定函数的参数,以计算成本和梯度。因为我们使用的是Python,所以我们可以使用SciPy的优化API来做同样的事情。 import scipy.optimize as opt result = opt.fmin_tnc(func=cost, x0=theta, fprime=gradient, args=(X, y)) cost(result[0], X, y) 0.20357134412164668 现在我们的数据集里有了最优模型参数,接下来我们要写一个函数,它使用我们训练过的参数theta来输出数据集X的预测,然后使用这个函数为我们分类器的训练精度打分。 def predict(theta, X): probability = sigmoid(X * theta.T) return [1 if x >= 0.5 else 0 for x in probability] theta_min = np.matrix(result[0]) predictions = predict(theta_min, X) correct = [1 if ((a == 1 and b == 1) or (a == 0 and b == 0)) else 0 for (a, b) in zip(predictions, y)] accuracy = (sum(map(int, correct)) % len(correct)) print 'accuracy = {0}%'.format(accuracy) accuracy = 89% 我们的逻辑回归分类器预测学生是否被录取的准确性可以达到89%,这是在训练集中的精度。我们没有保留一个hold-out set或使用交叉验证来获得准确的近似值,所以这个数字可能高于实际的值。 正则化逻辑回归 既然我们已经有了逻辑回归的工作实现,我们将通过添加正则化来改善算法。正则化是成本函数的一个条件,使算法倾向于更简单的模型(在这种情况下,模型会减小系数),原理就是帮助减少过度拟合和帮助模型提高通用化能力。我们使用逻辑回归的正则化版本去解决稍带挑战性的问题, 想象你是工厂的产品经理,你有一些芯片在两种不同测试上的测试结果。通过两种测试,你将会决定那种芯片被接受或者拒绝。为了帮助你做这个决定,你将会有以往芯片的测试结果数据集,并且通过它建立一个逻辑回归模型。 现在可视化数据。 path = os.getcwd() + '\data\ex2data2.txt' data2 = pd.read_csv(path, header=None, names=['Test 1', 'Test 2', 'Accepted']) positive = data2[data2['Accepted'].isin([1])] negative = data2[data2['Accepted'].isin([0])] fig, ax = plt.subplots(figsize=(12,8)) ax.scatter(positive['Test 1'], positive['Test 2'], s=50, c='b', marker='o', label='Accepted') ax.scatter(negative['Test 1'], negative['Test 2'], s=50, c='r', marker='x', label='Rejected') ax.legend() ax.set_xlabel('Test 1 Score') ax.set_ylabel('Test 2 Score') 这个数据看起来比以前的例子更复杂,你会注意到没有线性决策线,数据也执行的很好,处理这个问题的一种方法是使用像逻辑回归这样的线性技术,就是构造出由原始特征多项式派生出来的特征。我们可以尝试创建一堆多项式特性以提供给分类器。 degree = 5 x1 = data2['Test 1'] x2 = data2['Test 2'] data2.insert(3, 'Ones', 1) for i in range(1, degree): for j in range(0, i): data2['F' + str(i) + str(j)] = np.power(x1, i-j) * np.power(x2, j) data2.drop('Test 1', axis=1, inplace=True) data2.drop('Test 2', axis=1, inplace=True) data2.head() 现在我们需要去修改成本和梯度函数以包含正则项。在这种情况下,将正则化矩阵添加到之前的计算中。这是更新后的成本函数。 def costReg(theta, X, y, learningRate): theta = np.matrix(theta) X = np.matrix(X) y = np.matrix(y) first = np.multiply(-y, np.log(sigmoid(X * theta.T))) second = np.multiply((1 - y), np.log(1 - sigmoid(X * theta.T))) reg = (learningRate / 2 * len(X)) * np.sum(np.power(theta[:,1:theta.shape[1]], 2)) return np.sum(first - second) / (len(X)) + reg 我们添加了一个名为“reg”的新变量,它是参数值的函数。随着参数越来越大,对成本函数的惩罚也越来越大。我们在函数中添加了一个新的“learning rate”参数。 这也是等式中正则项的一部分。 learning rate为我们提供了一个新的超参数,我们可以使用它来调整正则化在成本函数中的权重。 接下来,我们将在梯度函数中添加正则化。 def gradientReg(theta, X, y, learningRate): theta = np.matrix(theta) X = np.matrix(X) y = np.matrix(y) parameters = int(theta.ravel().shape[1]) grad = np.zeros(parameters) error = sigmoid(X * theta.T) - y for i in range(parameters): term = np.multiply(error, X[:,i]) if (i == 0): grad[i] = np.sum(term) / len(X) else: grad[i] = (np.sum(term) / len(X)) + ((learningRate / len(X)) * theta[:,i]) return grad 与成本函数一样,将正则项加到最初的计算中。与成本函数不同的是,我们包含了确保第一个参数不被正则化的逻辑。这个决定背后的直觉是,第一个参数被认为是模型的“bias”或“intercept”,不应该被惩罚。 我们像以前那样测试新函数 # set X and y (remember from above that we moved the label to column 0) cols = data2.shape[1] X2 = data2.iloc[:,1:cols] y2 = data2.iloc[:,0:1] # convert to numpy arrays and initalize the parameter array theta X2 = np.array(X2.values) y2 = np.array(y2.values) theta2 = np.zeros(11) learningRate = 1 costReg(theta2, X2, y2, learningRate) 0.6931471805599454 我们能使用先前的最优代码寻找最优模型参数。 result2 = opt.fmin_tnc(func=costReg, x0=theta2, fprime=gradientReg, args=(X2, y2, learningRate)) result2 (数组([ 0.35872309, -3.22200653, 18.97106363, -4.25297831, 18.23053189, 20.36386672, 8.94114455, -43.77439015, -17.93440473, -50.75071857, -2.84162964]), 110, 1) 最后,我们可以使用前面应用的相同方法,为训练数据创建标签预测,并评估模型的性能。 theta_min = np.matrix(result2[0]) predictions = predict(theta_min, X2) correct = [1 if ((a == 1 and b == 1) or (a == 0 and b == 0)) else 0 for (a, b) in zip(predictions, y2)] accuracy = (sum(map(int, correct)) % len(correct)) print 'accuracy = {0}%'.format(accuracy) 准确度 = 91%

珍宝珠 2019-12-02 03:22:33 0 浏览量 回答数 0

问题

E-MapReduce HadoopPig 开发手册是什么?

nicenelly 2019-12-01 21:17:44 1567 浏览量 回答数 1

问题

以链接列表的形式构建字典并出现分段错误(核心转储)错误

kun坤 2019-12-01 22:06:53 3 浏览量 回答数 1

问题

一些书签小程序不能在iOS上执行:如何调试?

游客5akardh5cojhg 2019-12-23 18:33:51 0 浏览量 回答数 0
阿里云大学 云服务器ECS com域名 网站域名whois查询 开发者平台 小程序定制 小程序开发 国内短信套餐包 开发者技术与产品 云数据库 图像识别 开发者问答 阿里云建站 阿里云备案 云市场 万网 阿里云帮助文档 免费套餐 开发者工具 企业信息查询 小程序开发制作 视频内容分析 企业网站制作 视频集锦 代理记账服务 企业建站模板