DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-11 is/are rejected under 35 U.S.C. 102(a)(1)/(a)(2) as being anticipated by Baik et al1 (“Baik”).
Regarding claim 1, Baik teaches an image processing support device (note that an “image processing support device” is addressed by the rejection of each element below where “support” is not specifically further limited in the body of the claim such that any device such as comprised below is considered to constitute an image processing support device; thus see Baik, paragraphs 0041-0046 and figure 3 teaching “frame processing apparatus includes a storage unit 300, a load calculation unit 310, a core selecting unit 320, and a frame generating unit 330” and “frame generating unit 330 may generate the frame that is to be displayed on the user interface, by processing the operation at the selected core using the optimum combination of cores 321” where such frames are images as they are “displayed” as will be further discussed below) comprising:
processing circuitry (see Baik, paragraphs 0041-0046 and figure 3 as explained above teaching “the frame processing apparatus includes a storage unit 300, a load calculation unit 310, a core selecting unit 320, and a frame generating unit 330” and “frame processing apparatus 300 may be or may included in a terminal, such as a mobile terminal, a computer, a smart phone, a MP3 player” such apparatus comprises processing circuitry such as included in “a terminal” where paragraphs 0077-0080 teach such apparatus as processing circuitry such as “The described hardware devices may be configured to act as one or more software modules that are recorded, stored, or fixed in one or more computer-readable storage media, in order to perform the operations and methods described above, or vice versa”)
to acquire, by taking each of a plurality of image processes as a target image process, an engine characteristic indicating a calculation formula to calculate calculation information being at least either of a used memory amount and a processing time in a case in which the target image process is performed with respect to each of one or more engines that can be used in the target image process among one or more engines corresponding to each of a plurality of types of processors, the calculation formula using information on image data being a processing target as an input (here the claim requires acquiring an “engine characteristic” which corresponds to some base mathematical formula that uses “information on image data” to calculate either memory or processing time for specific algorithms which correspond to various “engines” that can be used in a “target image process” which corresponds to the image processing operation that is to be performed by one of the processors; thus see Baik, paragraphs 0047-0052 and figure 4 teaching “a process of calculating a system load that is used to display a frame” where for example “frame N includes four operations 1, 2, 3 and 4, and loads of a system may be calculated for each operation” where “each operation has an operation type 401 and a view area 402” and “operation type may refer to a characteristic of operation in terms of the amount of computation. For example, ‘Op1’ may represent ‘fill’ and ‘Op2’ may represent ‘alpha blend’” such that here the different operations are a plurality of target image processes and “A load for each operation may be calculated for each type of the operation by an equation that is shown on top of a lower table in FIG. 4, based on the operation type and the view area of the operation. That is, a load for processing the entire frame is not calculated as a whole, but is instead calculated based on individual operations” such that this formula is an engine characteristic indicating a calculation formula to calculate a processing time with respect to the engine characteristic being used in the target process by each of a plurality of types of processors as further explained in paragraphs 0053-0057 and figure 5 teaching “processing capabilities and power consumptions that are mapped to respective cores of a system” where “processing capability represents the number of pixels that are processed for a predetermined update time, for example, 16 ms. The power consumption represents power consumption to process a drawing operation that is performed on a predetermined number of pixels, for example, 100 megapixels” such that the load calculation based on the image data establishes a base formula corresponding to the type of engine operation which is to calculate a processing time with respect to each processing unit as further explained below);
to acquire, by taking each of a plurality of apparatuses as a target apparatus, apparatus characteristic information indicating a correction content for the calculation formula indicated by the engine characteristic with respect to one or more types of processors that are included in the target apparatus among the plurality of types of processors (see Baik, paragraphs 0053-0057 and figure 5A teaching apparatus characteristic information indicating a correction content for the calculation formula indicated by the engine characteristic as where “Processing capabilities and power consumptions to process operations corresponding to operation type Op1, Op2 and Op3 are calculated for each of cores CPU0, CPU1 and GPU as shown in FIG. 5A” and “processing capability represents the number of pixels that are processed for a predetermined update time, for example, 16 ms. The power consumption represents power consumption to process a drawing operation that is performed on a predetermined number of pixels, for example, 100 megapixels” where “higher the capability value the less power consumption used to process a predetermined number of pixels. Accordingly, a core having a higher capability value may be primarily selected during the processing of operations” such that a generic or base load corresponding to a respective engine and type of image processing operation and a number of pixel operations of a certain type has been established corresponding to the processing time, where greater loads correspond to greater processing time, and the “capability” and “power” information are characteristic information indicating a correction content for the calculation formula where the “processing capability” acts as the correction content which corrects the base pixel load that will be processed over time with respect to the various processing units which translates the load based on the time-based processing capability of each processor), and
to calculate the calculation information in a case in which a designated process being an image process designated among the plurality of image processes is performed by a designated apparatus being an apparatus designated among the plurality of apparatuses, by taking each of one or more engines that can be used in the designated process, and corresponds to a processor included in the designated apparatus as an engine being a calculation target, based on the engine characteristic and the apparatus characteristic information with respect to the engine being the calculation target (here the claim requires to actually calculate the calculation information which may be an amount of memory or processing time to perform an operation with respect to a specifically selected or designated image process and apparatus among the plurality of operations and apparatuses; thus see Baik, paragraphs 0068-0076 and figure 7 teaching “the processing capability of the selected core may be compared with a total load that is obtained by performing a summation of the loads, which are calculated for each operation type of the operations. If the processing capability of the selected core is lower than the total load, the selected core may be determined to have no ability to handle the loads” and “If a result of the determination is that the selected core has no ability to handle the loads of all of the operations, whether there is an additional core available to process the load is determined, in 730. If there are additional cores available to process the load, a core that has a highest priority is selected from among the cores and the selected core is additionally allocated to process the load, in 740” where “a capability value of each core existing in the multicore-system may be calculated based on the number of pixels of an operation processed for each core and a power consumption for each core to process a predetermined number of pixels. Thereafter, a core having a maximum capability value or a minimum capability value may be selected from among the cores existing in the multicore-system as the additional core” and “operations 720, 730, and 740 may be repeated until enough cores are selected to handle the load of processing the frame” and “If a result of the determination in 720 is that the selected cores have an ability to handle the load of all of the operations forming the frame, the loads are allocated such that the operations are processed through the selected cores, in 750” such that the calculation information such as the processing time for each specific core is determined when doing core selection when determining the capability value according to the “processing capability” of each core).
Regarding claim 2, Baik teaches all that is required as applied to claim 1 above and further teaches wherein the processing circuitry applies the correction content indicated by the apparatus characteristic information to the calculation formula indicated by the engine characteristic, and calculates the calculation information (see Baik, paragraphs 0068-0075 and figure 7 as explained above where the calculation information is actually calculated with respect to the operations for the processors where “the processing capability of the selected core may be compared with a total load that is obtained by performing a summation of the loads, which are calculated for each operation type of the operations” where for example “If the processing capability of the selected core is lower than the total load, the selected core may be determined to have no ability to handle the loads” and “For example, a capability value of each core existing in the multicore-system may be calculated based on the number of pixels of an operation processed for each core and a power consumption for each core to process a predetermined number of pixels. Thereafter, a core having a maximum capability value or a minimum capability value may be selected from among the cores existing in the multicore-system as the additional core”).
Regarding claim 3, Baik, teaches all that is required as applied to claim 2 above and further teaches wherein the processing circuitry enters the information on the image data being the processing target in the calculation formula to which the correction content is applied, and calculates the calculation information (see Baik, paragraphs 0047-0052 and figure 4 where the information on the image data being the processing target corresponds to the “operation type” and “view area” and “pixels” of the view area corresponding to the operation and as in paragraphs 0068-0075 such entering and calculating occurs in order to perform the core selection of where to send the information for processing).
Regarding claim 4, Baik teaches all that is required as applied to claim 1 above and further teaches wherein the processing circuitry calculates the calculation information with respect to each of two or more designated processes in a case in which two or more image processes are designated as the designated process (see Baik, paragraphs 0047-0052 where “each operation” corresponds to a designated image process such as “four operations 1, 2, 3 and 4” and “’Op1’ may represent ‘fill’ and ‘Op2’ may represent ‘alpha blend’” which are image processes).
Regarding claim 5, Baik, as rendered definite as explained above, Baik teaches all that is required as applied to claim 1 above and further teaches wherein the processing circuitry adds, in a case in which the processing time is calculated as the calculation information, and the designated process is performed in parallel for a number of the designated process performed in parallelsee Baik, Abstract, teaching “perform a user interface drawing in a multi-core environment, an optimum combination of hardware components capable of operating with the minimum of power consumption while satisfying a requirement of a user may be obtained and a parallel user interface drawing may be performed by use of the optimum combination of hardware components” such that the multi-core processing units are performing designated draw processes in parallel as further explained in paragraph 0005 describing the technological environment and field which the invention operates in and seeks to improve where “a draw process is performed to generate a display on a user interface (UI) through a parallelization of operations, hardware for a drawing operation is initially allocated such that a frame is divided for each hardware, the frame for each hardware is allocated for each area, and each area is processed at the same time” and prior art suffered from “a load required for a draw process is not identified. Accordingly, even though a plurality of additional hardware components are available for use in the parallelization of operations, the same combination of initially allocated hardware components are used each time a frame is processed” and this parallel processing may be improved through the multi-core selection based on the designated processes and their processing times as explained above, where then further as explained in paragraphs 0047-0052 and figure 4 the designated process such as “Op1” or “Op2” or “Op3” or “Op3” which may have different designated process types, are performed in parallel and a correction factor is added for “number of OP-1” times an “alpha” factor which corresponds to how many instances of that specification operation are being executed simultaneously to draw that frame in parallel and thus adds this to the processing time load as this introduces a recognition that performing a greater number of operations in parallel has other processing costs beyond the base operation calculation).
Regarding claim 6, Baik teaches all that is required as applied to claim 1 above and further teaches wherein the processing circuitry specifies an engine of which the calculation information calculated satisfies a limiting value, with respect to the designated process (see Baik, paragraph 0006 teaching a goal of the system to “satisfy a desired response time” and as in paragraphs 0053-0057 a predetermined response time may correspond to the processing capability such that the proper core is selected to satisfy such a desired response time and as in paragraphs 0068-0075 then when the core is selected for processing the data with respect to the designated process this specifies the core to act as an engine of which the calculation information satisfies this limiting value in order to provide the frame in the proper response time).
Regarding claim 7, Baik teaches all that is required as applied to claim 1 above and further teaches wherein the processing circuitry specifies an engine of which the calculation information calculated is best, with respect to the designated process (see Baik, paragraphs 0053-0057 teaching a metric corresponding to which is best where “higher the capability value the less power consumption used to process a predetermined number of pixels. Accordingly, a core having a higher capability value may be primarily selected during the processing of operations” and as in paragraphs 0068-0075 the optimal cores are selected based on such capability and “a core that has a highest priority is selected from among the cores and the selected core is additionally allocated to process the load” such that this specifies the engine with respect to the designated process where that is the best engine for the process).
Regarding claim 8, Baik teaches all that is required as applied to claim 6 above and further teaches wherein the processing circuitry generates an execution module to cause the engine specified to perform the designated process (see Baik, paragraphs 0068-0075 and figure 7 teaching “a result of the determination in 720 is that the selected cores have an ability to handle the load of all of the operations forming the frame, the loads are allocated such that the operations are processed through the selected cores, in 750” such that this generates an execution model allocating the loads to the cores for execution to draw the frame according to the designated processes).
Regarding claim 9, the instant claim recites an “image processing support method comprising: by a computer” performing the various functions performed by the device functioning as a computer as recited in claim 1. Baik teaches such a method performed by a computer as explained above. Therefore, the limitations of claim 9 correspond to the limitations of claim 1; thus it is rejected on the same grounds as claim 1.
Regarding claim 10, the instant claim recites an apparatus in the form of a non-transitory computer readable medium storing an image processing support program to cause a computer to function as an image processing support device to perform the functions as performed by the device of claim 1. Baik teaches such a device and its functions as explained in the rejection of claim 1 and as embodied in the form of a non-transitory computer readable medium storing an image processing support program to cause a computer to function as an image processing support device (see Baik, paragraphs 0041-0046 and figure 3 as explained above teaching “the frame processing apparatus includes a storage unit 300, a load calculation unit 310, a core selecting unit 320, and a frame generating unit 330” and “frame processing apparatus 300 may be or may included in a terminal, such as a mobile terminal, a computer, a smart phone, a MP3 player” such apparatus comprises processing circuitry such as included in “a terminal” where paragraphs 0077-0080 teach such apparatus as processing circuitry such as “The described hardware devices may be configured to act as one or more software modules that are recorded, stored, or fixed in one or more computer-readable storage media, in order to perform the operations and methods described above, or vice versa”).
Regarding claim 11, Baik teaches all that is required as applied to claim 1 above and further teaches wherein the image data includes a width, a height, and a number of bits per pixel (note that the limitation modifies “the calculation formula using information on image data being a processing target as an input” such that the limitation would be interpreted as the calculation formula using information on image data including a width, a height, and a number of bits per pixel being a processing target as an input meaning that the calculation formula uses “information on” such image data as an input such that the width, height and a number of bits per pixel are not required as actual inputs to the calculation formula as recited and as can be seen such quantities are not utilized by any other aspect of the claims; further note that an image having “a number of bits per pixel” does require that the image be one represented in pixel format and there must be any number of bits per pixel requiring some bit representation or digital representation of the image, but it should be understood that a number of bits per pixel does not require any specific value, any specific number of bits per pixel, any specific format of the pixels, a specific bit depth or the like; thus see Baik, paragraphs 0047-0056 where the load calculation is calculated with respect to information on or about a particular image made up of pixels and the operations that are to be performed on specific areas and pixels of the images defined by the view area of the image that the operation will be applied to and here the load calculation is for based on information about the frame of image data and its pixels as in figure 4 teaching "an example of a process of calculating a system load that is used to display a frame (Frame N)" and "frame N includes four operations 1, 2, 3 and 4, and loads of a system may be calculated for each operation. In the example of FIG. 4, each operation has an operation type 401 and a view area 402" and "operation 1 has an operation type of Op1 and a view area of 20 pixels, operation 2 has an operation type of Op2 and a view area of 60 pixels, operation 3 has an operation type of Op2 and a view area of 60 pixels, and operation 4 has an operation type of Op3 and a view area of 40 pixels" and "load for each operation may be calculated for each type of the operation by an equation that is shown on top of a lower table in FIG. 4, based on the operation type and the view area of the operation. That is, a load for processing the entire frame is not calculated as a whole, but is instead calculated based on individual operations" such that here the image data corresponds to the frame N of data which as can be seen in figure 4 has a width and a height, and as the image is digitally represented by pixels and processed by the digital processing system, such image inherently has a number of bits per pixel as in order for a pixel to be represented to a computer digitally in the system disclosed, it must be expressed with respect to some number of bits necessarily; note that such information of height, width and a number of bits per pixel is not required to be an input to the calculation formula, such that Baik as explained above uses the information on the image data such as operations to be performed on the image data and view areas of the pixel data relating to the operations and does not need to teach that such information is specifically an input to the calculation formula as this is not required).
Response to Arguments
Applicant's arguments filed 7/2/2026 have been fully considered but they are not persuasive. Applicant argues on page 7 of “REMARKS” filed 7/2/2026, that Baik fails to teach “the calculation formula using information on image data being a processing target as an input.” The Examiner respectfully disagrees. The calculation of the power consumption may be based on a "predetermined number of pixels" but it is still with regard to the specific image input and operations on the pixels and view area such that the power consumption is still based on the input image which determines the operation type, which then means that the power consumption is determined for that specific image for that operation and view area as that is the load of the operation. The processing capability is not based on a predetermined number of pixels, but rather is determined for each operation and operation type where for example a cpu1 may be capable of processing a type of operation such as an op1 fill operation at a certain number of pixels per second over some time period (see Baik, paragraphs 0047-0056 where the load is calculated with respect to a particular image and the operations that are to be performed on specific areas of the images defined by the view area of the image that the operation will be applied to where the load calculation is for a frame of data and not a general load for an operation as in figure 4 teaching "an example of a process of calculating a system load that is used to display a frame (Frame N)" and "frame N includes four operations 1, 2, 3 and 4, and loads of a system may be calculated for each operation. In the example of FIG. 4, each operation has an operation type 401 and a view area 402" and "operation 1 has an operation type of Op1 and a view area of 20 pixels, operation 2 has an operation type of Op2 and a view area of 60 pixels, operation 3 has an operation type of Op2 and a view area of 60 pixels, and operation 4 has an operation type of Op3 and a view area of 40 pixels" and "load for each operation may be calculated for each type of the operation by an equation that is shown on top of a lower table in FIG. 4, based on the operation type and the view area of the operation. That is, a load for processing the entire frame is not calculated as a whole, but is instead calculated based on individual operations" and paragraph 0065 teaching "each of the loads of the system may be quantified based on the operation type and the number of pixels of the view area") such that this determines for example a time to process a pixel of this operation type for the number of pixels corresponding to the view area of the input image that is to be processed, and the greater number of pixels that can be processed by a certain device for a particular operation in a time period yields a greater capability value in relation to those operations in relation to that input image data. Accordingly, Applicant’s characterization of Baik as generically determining calculation information is not persuasive. Thus the underlying load calculation for the capability corresponds to the calculation information, and the different processors characteristics influence the calculation information such that the different processors for the operations have different capabilities calculated for the same operation on the same pixels. Thus Baik does disclose the calculation formula using information on image data being a processing target as an input and Applicant's argument to the contrary is not persuasive. Claims 9 and 10 remain rejected for the same reasons as claim 1.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See Kawano et al (US PGPUB No. 20160127604) teaching a system which provides a base formula for calculating load operations for image processing tasks to be assigned to different processing engines/cores with regard to a certain amount of time and determining a correction factor in relation to engine characteristics that will carry out the processing (see Kawana, paragraphs 0054-0061 teaching “processing load estimator 103, the computing performance estimator 104, and the core selection processing portion 105 perform processing for selecting a core 1CR which is to perform the rasterization process” and “processing load estimator 103 estimates, for each page, a load R required for the rasterization process to be applied to the intermediate language data 52” and “the processing load estimator 103 calculates (estimates) a load R based on the following equation (1)” and “computing performance estimator 104 calculates (estimates) the computing performance (processing capability) of each of the three cores 1CR” where “Information on specifications of each of the cores 1CR is prepared in advance in the form of table data. The information is, for example, a computing time taken to execute predetermined processing. The computing performance estimator 104 then calculates computing performance by substituting the information into a predetermined function” and “core selection processing portion 105 makes the first-order core 1CR (a first core 1CR1 in this example) as a target (Step #701 and Step #702). The core selection processing portion 105 estimates a time Tj taken for the first core 1CR1 to perform rasterization on the first page (a time T1 herein) (Step #703). The estimation may be made based on the following equation (2)” and “core selection processing portion 105 compares the time Tj (the time T1 herein) with a threshold D” and paragraphs 0076-0082 teaching “processing load estimator 103 estimates (calculates) a load R required for the processing” and “Functions or tables used to calculate the load R are prepared, in advance, depending on algorithms of the processing of the three cases. The processing load estimator 103 estimates a load R by using the functions or tables instead of using the foregoing equation (1)” and “computing performance estimator 104 estimates the computing performance of each of the cores 1CR as with the case of rasterization. The computing performance estimator 104 may estimate the performance depending on processing to be executed by the selected core 1CR. Stated differently, the computing performance estimator 104 may estimate a time taken for the processing to be performed on data of a regular image” and “the core selection processing portion 105 selects a core 1CR which is to execute processing based on the results of estimation (computation) by the processing load estimator 103 and the computing performance estimator 104. Note that the coefficient b in the equation (2) is preset depending on the processing type and so on. The same is similarly applied to the threshold D. As described earlier, the threshold D is so preset that the speed of a device outputting an image or data thereof is utilized” such that here a correction factor such as “b” is applied with respect to the calculation formula of the image processing task in a similar manner to the system recited in claim 1).
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SCOTT E SONNERS whose telephone number is (571)270-7504. The examiner can normally be reached Mon-Friday 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Wu can be reached at (571) 272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SCOTT E SONNERS/Examiner, Art Unit 2613
/XIAO M WU/Supervisory Patent Examiner, Art Unit 2613
1 US PGPUB No. 20120154410
Prosecution Insights